Note: This is test shard 2 of 8.
[==========] Running 8 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 1 test from RebalanceIgnoredTserversTest
[ RUN ] RebalanceIgnoredTserversTest.Basic
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240502 13:59:39.162860 26726 test_util.cc:268] Using random seed: 60417242
I20240502 13:59:39.170835 26726 ts_itest-base.cc:112] Starting cluster with:
I20240502 13:59:39.170902 26726 ts_itest-base.cc:113] --------------
I20240502 13:59:39.170918 26726 ts_itest-base.cc:114] 5 tablet servers
I20240502 13:59:39.170940 26726 ts_itest-base.cc:115] 3 replicas per TS
I20240502 13:59:39.170964 26726 ts_itest-base.cc:116] --------------
2024-05-02T13:59:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T13:59:39Z Disabled control of system clock
I20240502 13:59:39.179948 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.25.190:34329
--webserver_interface=127.26.25.190
--webserver_port=0
--builtin_ntp_servers=127.26.25.148:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.25.190:34329
--enable_range_replica_placement=false with env {}
W20240502 13:59:39.282063 26738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:39.282290 26738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:39.282357 26738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:39.285364 26738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 13:59:39.285427 26738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:39.285461 26738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 13:59:39.285493 26738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 13:59:39.288995 26738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.26.25.190:34329
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.25.190:34329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.26.25.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.26738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:39.289834 26738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:39.290900 26738 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:39.297502 26744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:39.298034 26746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:39.298553 26743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:39.299335 26738 server_base.cc:1034] running on GCE node
I20240502 13:59:39.299799 26738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:39.300694 26738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:39.301846 26738 hybrid_clock.cc:648] HybridClock initialized: now 1714658379301834 us; error 25 us; skew 500 ppm
I20240502 13:59:39.308565 26738 webserver.cc:460] Webserver started at http://127.26.25.190:37459/ using document root <none> and password file <none>
I20240502 13:59:39.309106 26738 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:39.309172 26738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:39.309360 26738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:39.310968 26738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "62094de2408148f9968bd40c86219ca1"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.311434 26738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "62094de2408148f9968bd40c86219ca1"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.314770 26738 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240502 13:59:39.317020 26752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:39.317956 26738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:39.318068 26738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
uuid: "62094de2408148f9968bd40c86219ca1"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.318152 26738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:39.335194 26738 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:39.335844 26738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:39.336002 26738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:39.342336 26738 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.190:34329
I20240502 13:59:39.342344 26804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.190:34329 every 8 connection(s)
I20240502 13:59:39.343266 26738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 13:59:39.344678 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 26738
I20240502 13:59:39.344825 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 13:59:39.345712 26805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:39.350847 26805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1: Bootstrap starting.
I20240502 13:59:39.352994 26805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:39.353806 26805 log.cc:824] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:39.355546 26805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1: No bootstrap required, opened a new log
I20240502 13:59:39.358400 26805 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62094de2408148f9968bd40c86219ca1" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 34329 } }
I20240502 13:59:39.358582 26805 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:39.358659 26805 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62094de2408148f9968bd40c86219ca1, State: Initialized, Role: FOLLOWER
I20240502 13:59:39.359072 26805 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62094de2408148f9968bd40c86219ca1" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 34329 } }
I20240502 13:59:39.359200 26805 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:39.359263 26805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:39.359367 26805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:39.360112 26805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62094de2408148f9968bd40c86219ca1" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 34329 } }
I20240502 13:59:39.360401 26805 leader_election.cc:302] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62094de2408148f9968bd40c86219ca1; no voters:
I20240502 13:59:39.360656 26805 leader_election.cc:288] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:39.360965 26810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:39.361137 26810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [term 1 LEADER]: Becoming Leader. State: Replica: 62094de2408148f9968bd40c86219ca1, State: Running, Role: LEADER
I20240502 13:59:39.361460 26810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62094de2408148f9968bd40c86219ca1" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 34329 } }
I20240502 13:59:39.361793 26805 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 13:59:39.362920 26811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62094de2408148f9968bd40c86219ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62094de2408148f9968bd40c86219ca1" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 34329 } } }
I20240502 13:59:39.363049 26811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:39.363394 26812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62094de2408148f9968bd40c86219ca1. Latest consensus state: current_term: 1 leader_uuid: "62094de2408148f9968bd40c86219ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62094de2408148f9968bd40c86219ca1" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 34329 } } }
I20240502 13:59:39.363451 26819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 13:59:39.363981 26812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1 [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:39.366489 26819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 13:59:39.370999 26819 catalog_manager.cc:1347] Generated new cluster ID: 9efe9b79f07345c5bc171bc8a9039135
I20240502 13:59:39.371057 26819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 13:59:39.387357 26819 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 13:59:39.388566 26819 catalog_manager.cc:1504] Loading token signing keys...
I20240502 13:59:39.409811 26819 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 62094de2408148f9968bd40c86219ca1: Generated new TSK 0
I20240502 13:59:39.410502 26819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 13:59:39.414700 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.129:0
--local_ip_for_outbound_sockets=127.26.25.129
--webserver_interface=127.26.25.129
--webserver_port=0
--tserver_master_addrs=127.26.25.190:34329
--builtin_ntp_servers=127.26.25.148:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20240502 13:59:39.534605 26829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:39.534850 26829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:39.534922 26829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:39.537955 26829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:39.538094 26829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.129
I20240502 13:59:39.541684 26829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.26.25.129
--webserver_port=0
--tserver_master_addrs=127.26.25.190:34329
--never_fsync=true
--heap_profile_path=/tmp/kudu.26829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.129
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:39.542531 26829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:39.543628 26829 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:39.551476 26835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:39.551594 26837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:39.552178 26834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:39.552770 26829 server_base.cc:1034] running on GCE node
I20240502 13:59:39.553105 26829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:39.553606 26829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:39.554766 26829 hybrid_clock.cc:648] HybridClock initialized: now 1714658379554759 us; error 27 us; skew 500 ppm
I20240502 13:59:39.556468 26829 webserver.cc:460] Webserver started at http://127.26.25.129:40679/ using document root <none> and password file <none>
I20240502 13:59:39.556984 26829 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:39.557044 26829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:39.557210 26829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:39.558789 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "bb06a516d33b4e5eb4b92499c564419c"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.559244 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "bb06a516d33b4e5eb4b92499c564419c"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.562443 26829 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240502 13:59:39.564617 26843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:39.565541 26829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:39.565641 26829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "bb06a516d33b4e5eb4b92499c564419c"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.565716 26829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:39.591027 26829 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:39.591647 26829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:39.591804 26829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:39.592329 26829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:39.593191 26829 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:39.593240 26829 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:39.593297 26829 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:39.593328 26829 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:39.601377 26829 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.129:39611
I20240502 13:59:39.601406 26957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.129:39611 every 8 connection(s)
I20240502 13:59:39.602241 26829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 13:59:39.610128 26958 heartbeater.cc:347] Connected to a master server at 127.26.25.190:34329
I20240502 13:59:39.610327 26958 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:39.610802 26958 heartbeater.cc:511] Master 127.26.25.190:34329 requested a full tablet report, sending...
I20240502 13:59:39.610882 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 26829
I20240502 13:59:39.610980 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 13:59:39.612290 26769 ts_manager.cc:194] Registered new tserver with Master: bb06a516d33b4e5eb4b92499c564419c (127.26.25.129:39611)
I20240502 13:59:39.613197 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.130:0
--local_ip_for_outbound_sockets=127.26.25.130
--webserver_interface=127.26.25.130
--webserver_port=0
--tserver_master_addrs=127.26.25.190:34329
--builtin_ntp_servers=127.26.25.148:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 13:59:39.613492 26769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.129:45869
W20240502 13:59:39.715396 26961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:39.715617 26961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:39.715660 26961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:39.718699 26961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:39.718809 26961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.130
I20240502 13:59:39.722499 26961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.26.25.130
--webserver_port=0
--tserver_master_addrs=127.26.25.190:34329
--never_fsync=true
--heap_profile_path=/tmp/kudu.26961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.130
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:39.723423 26961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:39.724565 26961 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:39.732141 26969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:39.732611 26966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:39.733143 26967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:39.733930 26961 server_base.cc:1034] running on GCE node
I20240502 13:59:39.734246 26961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:39.734730 26961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:39.735893 26961 hybrid_clock.cc:648] HybridClock initialized: now 1714658379735863 us; error 37 us; skew 500 ppm
I20240502 13:59:39.737658 26961 webserver.cc:460] Webserver started at http://127.26.25.130:45871/ using document root <none> and password file <none>
I20240502 13:59:39.738220 26961 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:39.738276 26961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:39.738451 26961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:39.740105 26961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "e02f75b670d241e6b22c2e23c806944d"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.740566 26961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "e02f75b670d241e6b22c2e23c806944d"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.743647 26961 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20240502 13:59:39.745682 26975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:39.746562 26961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:39.746657 26961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "e02f75b670d241e6b22c2e23c806944d"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.746742 26961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:39.767997 26961 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:39.768728 26961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:39.768880 26961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:39.769418 26961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:39.770316 26961 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:39.770364 26961 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:39.770402 26961 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:39.770416 26961 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:39.778455 26961 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.130:35879
I20240502 13:59:39.778482 27089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.130:35879 every 8 connection(s)
I20240502 13:59:39.779366 26961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 13:59:39.787804 27090 heartbeater.cc:347] Connected to a master server at 127.26.25.190:34329
I20240502 13:59:39.788071 27090 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:39.788568 27090 heartbeater.cc:511] Master 127.26.25.190:34329 requested a full tablet report, sending...
I20240502 13:59:39.788739 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 26961
I20240502 13:59:39.788829 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 13:59:39.789615 26769 ts_manager.cc:194] Registered new tserver with Master: e02f75b670d241e6b22c2e23c806944d (127.26.25.130:35879)
I20240502 13:59:39.790319 26769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.130:53565
I20240502 13:59:39.790931 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.131:0
--local_ip_for_outbound_sockets=127.26.25.131
--webserver_interface=127.26.25.131
--webserver_port=0
--tserver_master_addrs=127.26.25.190:34329
--builtin_ntp_servers=127.26.25.148:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20240502 13:59:39.894693 27093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:39.894927 27093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:39.894989 27093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:39.897998 27093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:39.898094 27093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.131
I20240502 13:59:39.901500 27093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.26.25.131
--webserver_port=0
--tserver_master_addrs=127.26.25.190:34329
--never_fsync=true
--heap_profile_path=/tmp/kudu.27093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.131
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:39.902257 27093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:39.903277 27093 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:39.911196 27099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:39.911365 27098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:39.911907 27101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:39.912472 27093 server_base.cc:1034] running on GCE node
I20240502 13:59:39.912765 27093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:39.913237 27093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:39.914377 27093 hybrid_clock.cc:648] HybridClock initialized: now 1714658379914353 us; error 38 us; skew 500 ppm
I20240502 13:59:39.916146 27093 webserver.cc:460] Webserver started at http://127.26.25.131:38587/ using document root <none> and password file <none>
I20240502 13:59:39.916764 27093 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:39.916834 27093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:39.917065 27093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:39.919046 27093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "7f544c1892b6473b84d4cfdabf61f14a"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.922405 27093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "7f544c1892b6473b84d4cfdabf61f14a"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.925803 27093 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 13:59:39.928014 27107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:39.928998 27093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:39.929092 27093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "7f544c1892b6473b84d4cfdabf61f14a"
format_stamp: "Formatted at 2024-05-02 13:59:39 on dist-test-slave-sldb"
I20240502 13:59:39.929165 27093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:39.965781 27093 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:39.966480 27093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:39.966624 27093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:39.967128 27093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:39.968037 27093 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:39.968088 27093 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:39.968129 27093 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:39.968150 27093 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:39.976962 27093 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.131:32843
I20240502 13:59:39.976994 27220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.131:32843 every 8 connection(s)
I20240502 13:59:39.977942 27093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 13:59:39.986574 27221 heartbeater.cc:347] Connected to a master server at 127.26.25.190:34329
I20240502 13:59:39.986569 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 27093
I20240502 13:59:39.986660 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 13:59:39.986778 27221 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:39.987311 27221 heartbeater.cc:511] Master 127.26.25.190:34329 requested a full tablet report, sending...
I20240502 13:59:39.988323 26769 ts_manager.cc:194] Registered new tserver with Master: 7f544c1892b6473b84d4cfdabf61f14a (127.26.25.131:32843)
I20240502 13:59:39.988680 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.132:0
--local_ip_for_outbound_sockets=127.26.25.132
--webserver_interface=127.26.25.132
--webserver_port=0
--tserver_master_addrs=127.26.25.190:34329
--builtin_ntp_servers=127.26.25.148:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 13:59:39.988909 26769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.131:53115
W20240502 13:59:40.091701 27224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:40.091923 27224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:40.091966 27224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:40.094975 27224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:40.095081 27224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.132
I20240502 13:59:40.098603 27224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.26.25.132
--webserver_port=0
--tserver_master_addrs=127.26.25.190:34329
--never_fsync=true
--heap_profile_path=/tmp/kudu.27224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.132
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:40.099452 27224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:40.100528 27224 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:40.107960 27232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:40.108428 27229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:40.109223 27230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:40.109827 27224 server_base.cc:1034] running on GCE node
I20240502 13:59:40.110180 27224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:40.110713 27224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:40.111928 27224 hybrid_clock.cc:648] HybridClock initialized: now 1714658380111918 us; error 26 us; skew 500 ppm
I20240502 13:59:40.113744 27224 webserver.cc:460] Webserver started at http://127.26.25.132:44325/ using document root <none> and password file <none>
I20240502 13:59:40.114267 27224 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:40.114328 27224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:40.114504 27224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:40.116418 27224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "758da1dffcd84542b500fc457aa067c4"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-sldb"
I20240502 13:59:40.116935 27224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "758da1dffcd84542b500fc457aa067c4"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-sldb"
I20240502 13:59:40.120126 27224 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 13:59:40.122252 27238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.123340 27224 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:40.123443 27224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "758da1dffcd84542b500fc457aa067c4"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-sldb"
I20240502 13:59:40.123529 27224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:40.140697 27224 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:40.141314 27224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:40.141471 27224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:40.141979 27224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:40.142836 27224 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:40.142884 27224 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.142931 27224 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:40.142963 27224 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.151106 27224 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.132:41709
I20240502 13:59:40.151170 27353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.132:41709 every 8 connection(s)
I20240502 13:59:40.152034 27224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 13:59:40.153862 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 27224
I20240502 13:59:40.153967 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 13:59:40.156097 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.133:0
--local_ip_for_outbound_sockets=127.26.25.133
--webserver_interface=127.26.25.133
--webserver_port=0
--tserver_master_addrs=127.26.25.190:34329
--builtin_ntp_servers=127.26.25.148:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 13:59:40.160710 27354 heartbeater.cc:347] Connected to a master server at 127.26.25.190:34329
I20240502 13:59:40.160953 27354 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:40.161645 27354 heartbeater.cc:511] Master 127.26.25.190:34329 requested a full tablet report, sending...
I20240502 13:59:40.162768 26769 ts_manager.cc:194] Registered new tserver with Master: 758da1dffcd84542b500fc457aa067c4 (127.26.25.132:41709)
I20240502 13:59:40.163600 26769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.132:58753
W20240502 13:59:40.267134 27357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:40.267361 27357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:40.267428 27357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:40.271092 27357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:40.271235 27357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.133
I20240502 13:59:40.275311 27357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.25.133
--webserver_port=0
--tserver_master_addrs=127.26.25.190:34329
--never_fsync=true
--heap_profile_path=/tmp/kudu.27357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.133
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:40.276206 27357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:40.277283 27357 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240502 13:59:40.285518 27357 server_base.cc:1034] running on GCE node
W20240502 13:59:40.285691 27363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:40.286195 27362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:40.286641 27365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:40.286999 27357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:40.287528 27357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:40.288668 27357 hybrid_clock.cc:648] HybridClock initialized: now 1714658380288659 us; error 26 us; skew 500 ppm
I20240502 13:59:40.290340 27357 webserver.cc:460] Webserver started at http://127.26.25.133:32791/ using document root <none> and password file <none>
I20240502 13:59:40.290870 27357 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:40.290932 27357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:40.291102 27357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:40.292753 27357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "a3f8078f229c410e84253b2dcd90cee9"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-sldb"
I20240502 13:59:40.293196 27357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "a3f8078f229c410e84253b2dcd90cee9"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-sldb"
I20240502 13:59:40.296300 27357 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 13:59:40.298329 27371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.299257 27357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:40.299360 27357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "a3f8078f229c410e84253b2dcd90cee9"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-sldb"
I20240502 13:59:40.299437 27357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:40.309598 27357 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:40.310158 27357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:40.310300 27357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:40.310779 27357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:40.311594 27357 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:40.311709 27357 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.311805 27357 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:40.311895 27357 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.319875 27357 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.133:42515
I20240502 13:59:40.319903 27486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.133:42515 every 8 connection(s)
I20240502 13:59:40.320703 27357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 13:59:40.321108 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 27357
I20240502 13:59:40.321211 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 13:59:40.328941 27487 heartbeater.cc:347] Connected to a master server at 127.26.25.190:34329
I20240502 13:59:40.329165 27487 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:40.329640 27487 heartbeater.cc:511] Master 127.26.25.190:34329 requested a full tablet report, sending...
I20240502 13:59:40.330545 26769 ts_manager.cc:194] Registered new tserver with Master: a3f8078f229c410e84253b2dcd90cee9 (127.26.25.133:42515)
I20240502 13:59:40.331053 26769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.133:42765
I20240502 13:59:40.335510 26726 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 13:59:40.346714 26769 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55786:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 13:59:40.368130 27420 tablet_service.cc:1467] Processing CreateTablet for tablet d207d27d13e54bdda9946a91d116248c (DEFAULT_TABLE table=TestTable [id=931842289a5744a0abe15a551c84a8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:40.368139 26891 tablet_service.cc:1467] Processing CreateTablet for tablet d207d27d13e54bdda9946a91d116248c (DEFAULT_TABLE table=TestTable [id=931842289a5744a0abe15a551c84a8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:40.369084 27420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d207d27d13e54bdda9946a91d116248c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:40.369086 26891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d207d27d13e54bdda9946a91d116248c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:40.374230 27504 tablet_bootstrap.cc:492] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9: Bootstrap starting.
I20240502 13:59:40.374445 27286 tablet_service.cc:1467] Processing CreateTablet for tablet d207d27d13e54bdda9946a91d116248c (DEFAULT_TABLE table=TestTable [id=931842289a5744a0abe15a551c84a8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:40.375321 27286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d207d27d13e54bdda9946a91d116248c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:40.376096 27505 tablet_bootstrap.cc:492] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: Bootstrap starting.
I20240502 13:59:40.376425 27504 tablet_bootstrap.cc:654] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:40.377233 27504 log.cc:824] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:40.378034 27505 tablet_bootstrap.cc:654] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:40.378752 27505 log.cc:824] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:40.378971 27504 tablet_bootstrap.cc:492] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9: No bootstrap required, opened a new log
I20240502 13:59:40.379196 27504 ts_tablet_manager.cc:1367] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240502 13:59:40.379901 27508 tablet_bootstrap.cc:492] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4: Bootstrap starting.
I20240502 13:59:40.380489 27505 tablet_bootstrap.cc:492] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: No bootstrap required, opened a new log
I20240502 13:59:40.380672 27505 ts_tablet_manager.cc:1367] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 13:59:40.381839 27508 tablet_bootstrap.cc:654] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:40.382534 27508 log.cc:824] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:40.382899 27504 raft_consensus.cc:348] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } }
I20240502 13:59:40.383118 27504 raft_consensus.cc:374] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:40.383188 27504 raft_consensus.cc:729] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3f8078f229c410e84253b2dcd90cee9, State: Initialized, Role: FOLLOWER
I20240502 13:59:40.383652 27504 consensus_queue.cc:260] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } }
I20240502 13:59:40.383930 27508 tablet_bootstrap.cc:492] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4: No bootstrap required, opened a new log
I20240502 13:59:40.383874 27505 raft_consensus.cc:348] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } }
I20240502 13:59:40.384081 27505 raft_consensus.cc:374] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:40.384119 27508 ts_tablet_manager.cc:1367] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240502 13:59:40.384145 27505 raft_consensus.cc:729] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb06a516d33b4e5eb4b92499c564419c, State: Initialized, Role: FOLLOWER
I20240502 13:59:40.384511 27504 ts_tablet_manager.cc:1398] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 13:59:40.384532 27505 consensus_queue.cc:260] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } }
I20240502 13:59:40.384862 27487 heartbeater.cc:503] Master 127.26.25.190:34329 was elected leader, sending a full tablet report...
I20240502 13:59:40.385308 27505 ts_tablet_manager.cc:1398] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 13:59:40.385427 26958 heartbeater.cc:503] Master 127.26.25.190:34329 was elected leader, sending a full tablet report...
I20240502 13:59:40.387586 27508 raft_consensus.cc:348] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } }
I20240502 13:59:40.387790 27508 raft_consensus.cc:374] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:40.387873 27508 raft_consensus.cc:729] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 758da1dffcd84542b500fc457aa067c4, State: Initialized, Role: FOLLOWER
I20240502 13:59:40.388320 27508 consensus_queue.cc:260] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } }
I20240502 13:59:40.389017 27354 heartbeater.cc:503] Master 127.26.25.190:34329 was elected leader, sending a full tablet report...
I20240502 13:59:40.389138 27508 ts_tablet_manager.cc:1398] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
W20240502 13:59:40.402817 27355 tablet.cc:2369] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:40.508152 27511 raft_consensus.cc:482] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:40.508342 27511 raft_consensus.cc:504] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } }
I20240502 13:59:40.509469 27511 leader_election.cc:288] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3f8078f229c410e84253b2dcd90cee9 (127.26.25.133:42515), 758da1dffcd84542b500fc457aa067c4 (127.26.25.132:41709)
I20240502 13:59:40.514182 27306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d207d27d13e54bdda9946a91d116248c" candidate_uuid: "bb06a516d33b4e5eb4b92499c564419c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "758da1dffcd84542b500fc457aa067c4" is_pre_election: true
I20240502 13:59:40.514182 27440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d207d27d13e54bdda9946a91d116248c" candidate_uuid: "bb06a516d33b4e5eb4b92499c564419c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3f8078f229c410e84253b2dcd90cee9" is_pre_election: true
I20240502 13:59:40.514487 27306 raft_consensus.cc:2454] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb06a516d33b4e5eb4b92499c564419c in term 0.
I20240502 13:59:40.514487 27440 raft_consensus.cc:2454] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb06a516d33b4e5eb4b92499c564419c in term 0.
I20240502 13:59:40.514952 26844 leader_election.cc:302] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 758da1dffcd84542b500fc457aa067c4, bb06a516d33b4e5eb4b92499c564419c; no voters:
I20240502 13:59:40.515168 27511 raft_consensus.cc:2781] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:40.515242 27511 raft_consensus.cc:482] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:40.515272 27511 raft_consensus.cc:3037] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:40.516209 27511 raft_consensus.cc:504] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } }
I20240502 13:59:40.516551 27511 leader_election.cc:288] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [CANDIDATE]: Term 1 election: Requested vote from peers a3f8078f229c410e84253b2dcd90cee9 (127.26.25.133:42515), 758da1dffcd84542b500fc457aa067c4 (127.26.25.132:41709)
I20240502 13:59:40.516860 27440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d207d27d13e54bdda9946a91d116248c" candidate_uuid: "bb06a516d33b4e5eb4b92499c564419c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3f8078f229c410e84253b2dcd90cee9"
I20240502 13:59:40.516867 27306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d207d27d13e54bdda9946a91d116248c" candidate_uuid: "bb06a516d33b4e5eb4b92499c564419c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "758da1dffcd84542b500fc457aa067c4"
I20240502 13:59:40.516974 27306 raft_consensus.cc:3037] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:40.516999 27440 raft_consensus.cc:3037] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:40.517793 27306 raft_consensus.cc:2454] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb06a516d33b4e5eb4b92499c564419c in term 1.
I20240502 13:59:40.517849 27440 raft_consensus.cc:2454] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb06a516d33b4e5eb4b92499c564419c in term 1.
I20240502 13:59:40.518083 26844 leader_election.cc:302] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 758da1dffcd84542b500fc457aa067c4, bb06a516d33b4e5eb4b92499c564419c; no voters:
I20240502 13:59:40.518294 27511 raft_consensus.cc:2781] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:40.518494 27511 raft_consensus.cc:686] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 1 LEADER]: Becoming Leader. State: Replica: bb06a516d33b4e5eb4b92499c564419c, State: Running, Role: LEADER
I20240502 13:59:40.518800 27511 consensus_queue.cc:237] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } }
I20240502 13:59:40.521735 26768 catalog_manager.cc:5484] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c reported cstate change: term changed from 0 to 1, leader changed from <none> to bb06a516d33b4e5eb4b92499c564419c (127.26.25.129). New cstate: current_term: 1 leader_uuid: "bb06a516d33b4e5eb4b92499c564419c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:40.548866 26726 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 13:59:40.550614 26726 ts_itest-base.cc:233] Waiting for 1 tablets on tserver bb06a516d33b4e5eb4b92499c564419c to finish bootstrapping
I20240502 13:59:40.556025 26726 ts_itest-base.cc:233] Waiting for 0 tablets on tserver e02f75b670d241e6b22c2e23c806944d to finish bootstrapping
I20240502 13:59:40.561164 26726 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 7f544c1892b6473b84d4cfdabf61f14a to finish bootstrapping
I20240502 13:59:40.566382 26726 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 758da1dffcd84542b500fc457aa067c4 to finish bootstrapping
I20240502 13:59:40.570887 26726 ts_itest-base.cc:233] Waiting for 1 tablets on tserver a3f8078f229c410e84253b2dcd90cee9 to finish bootstrapping
W20240502 13:59:40.571621 27488 tablet.cc:2369] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 13:59:40.603588 26959 tablet.cc:2369] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:40.747910 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:40.749930 27420 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:40.750281 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:40.750779 26891 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:40.762516 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:40.792234 27090 heartbeater.cc:503] Master 127.26.25.190:34329 was elected leader, sending a full tablet report...
I20240502 13:59:40.975956 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:40.976056 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:40.976779 26891 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:40.977221 27420 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:40.985102 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:40.990605 27221 heartbeater.cc:503] Master 127.26.25.190:34329 was elected leader, sending a full tablet report...
I20240502 13:59:40.993497 27511 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [LEADER]: Connected to new peer: Peer: permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:41.001111 27515 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:41.202906 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.203234 27420 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.204152 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.204509 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.219223 26891 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:41.423254 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.423938 27420 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.427431 26891 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:41.427677 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.428068 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.510443 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.512313 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.513200 27420 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.513639 26891 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:41.521992 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.613482 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.614441 27420 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.616115 26891 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:41.619992 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.621074 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.783291 26769 ts_manager.cc:292] Set tserver state for bb06a516d33b4e5eb4b92499c564419c to MAINTENANCE_MODE
I20240502 13:59:41.978240 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.979032 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.979622 26891 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:41.982223 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:41.986631 27420 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:42.074795 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:42.074882 27420 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:42.074986 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:42.076426 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:42.087703 26891 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:42.175957 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:42.177160 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:42.177246 27420 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:42.179860 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:42.180366 26891 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:42.215682 26912 consensus_queue.cc:237] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } }
I20240502 13:59:42.217133 27440 raft_consensus.cc:1261] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 1 FOLLOWER]: Refusing update from remote peer bb06a516d33b4e5eb4b92499c564419c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:42.217133 27306 raft_consensus.cc:1261] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 1 FOLLOWER]: Refusing update from remote peer bb06a516d33b4e5eb4b92499c564419c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:42.217593 27838 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 13:59:42.217790 27679 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [LEADER]: Connected to new peer: Peer: permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 13:59:42.219995 27841 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } }
I20240502 13:59:42.220254 27440 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } }
I20240502 13:59:42.220261 27306 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } }
I20240502 13:59:42.222982 26769 catalog_manager.cc:5484] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 reported cstate change: config changed from index -1 to 2. New cstate: current_term: 1 leader_uuid: "bb06a516d33b4e5eb4b92499c564419c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } }
I20240502 13:59:42.226858 26912 consensus_queue.cc:237] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: true } }
I20240502 13:59:42.228358 27440 raft_consensus.cc:1261] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 1 FOLLOWER]: Refusing update from remote peer bb06a516d33b4e5eb4b92499c564419c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 13:59:42.228736 27679 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 13:59:42.229444 27306 raft_consensus.cc:1261] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 1 FOLLOWER]: Refusing update from remote peer bb06a516d33b4e5eb4b92499c564419c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 13:59:42.229808 27841 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [LEADER]: Connected to new peer: Peer: permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 13:59:42.230512 27838 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER e02f75b670d241e6b22c2e23c806944d (127.26.25.130) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: true } } }
I20240502 13:59:42.231711 27306 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER e02f75b670d241e6b22c2e23c806944d (127.26.25.130) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: true } } }
I20240502 13:59:42.232463 26756 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet d207d27d13e54bdda9946a91d116248c with cas_config_opid_index 2: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 13:59:42.233690 26768 catalog_manager.cc:5484] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c reported cstate change: config changed from index 2 to 3, NON_VOTER e02f75b670d241e6b22c2e23c806944d (127.26.25.130) added. New cstate: current_term: 1 leader_uuid: "bb06a516d33b4e5eb4b92499c564419c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:42.234364 27440 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER e02f75b670d241e6b22c2e23c806944d (127.26.25.130) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: true } } }
I20240502 13:59:42.238126 26912 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d207d27d13e54bdda9946a91d116248c"
dest_uuid: "bb06a516d33b4e5eb4b92499c564419c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:36772
I20240502 13:59:42.238303 26912 raft_consensus.cc:595] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 1 LEADER]: Received request to transfer leadership
W20240502 13:59:42.238644 26846 consensus_peers.cc:485] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c -> Peer e02f75b670d241e6b22c2e23c806944d (127.26.25.130:35879): Couldn't send request to peer e02f75b670d241e6b22c2e23c806944d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d207d27d13e54bdda9946a91d116248c. This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:42.699537 27838 raft_consensus.cc:979] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: : Instructing follower a3f8078f229c410e84253b2dcd90cee9 to start an election
I20240502 13:59:42.699668 27838 raft_consensus.cc:1067] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 1 LEADER]: Signalling peer a3f8078f229c410e84253b2dcd90cee9 to start an election
I20240502 13:59:42.700286 27440 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d207d27d13e54bdda9946a91d116248c"
dest_uuid: "a3f8078f229c410e84253b2dcd90cee9"
from {username='slave'} at 127.26.25.129:41907
I20240502 13:59:42.700479 27440 raft_consensus.cc:482] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 13:59:42.700549 27440 raft_consensus.cc:3037] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:42.701344 27440 raft_consensus.cc:504] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: true } }
I20240502 13:59:42.702387 27440 leader_election.cc:288] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [CANDIDATE]: Term 2 election: Requested vote from peers bb06a516d33b4e5eb4b92499c564419c (127.26.25.129:39611), 758da1dffcd84542b500fc457aa067c4 (127.26.25.132:41709)
I20240502 13:59:42.705775 26912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d207d27d13e54bdda9946a91d116248c" candidate_uuid: "a3f8078f229c410e84253b2dcd90cee9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "bb06a516d33b4e5eb4b92499c564419c"
I20240502 13:59:42.705945 26912 raft_consensus.cc:3032] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 1 LEADER]: Stepping down as leader of term 1
I20240502 13:59:42.706005 26912 raft_consensus.cc:729] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb06a516d33b4e5eb4b92499c564419c, State: Running, Role: LEADER
I20240502 13:59:42.706195 26912 consensus_queue.cc:260] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: true } }
I20240502 13:59:42.706229 27306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d207d27d13e54bdda9946a91d116248c" candidate_uuid: "a3f8078f229c410e84253b2dcd90cee9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "758da1dffcd84542b500fc457aa067c4"
I20240502 13:59:42.706348 27306 raft_consensus.cc:3037] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:42.706394 26912 raft_consensus.cc:3037] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:42.707168 27306 raft_consensus.cc:2454] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3f8078f229c410e84253b2dcd90cee9 in term 2.
I20240502 13:59:42.707264 26912 raft_consensus.cc:2454] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3f8078f229c410e84253b2dcd90cee9 in term 2.
I20240502 13:59:42.707541 27372 leader_election.cc:302] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 758da1dffcd84542b500fc457aa067c4, a3f8078f229c410e84253b2dcd90cee9; no voters:
I20240502 13:59:42.707767 27842 raft_consensus.cc:2781] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 13:59:42.707958 27842 raft_consensus.cc:686] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 2 LEADER]: Becoming Leader. State: Replica: a3f8078f229c410e84253b2dcd90cee9, State: Running, Role: LEADER
I20240502 13:59:42.708268 27842 consensus_queue.cc:237] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: true } }
I20240502 13:59:42.710582 26768 catalog_manager.cc:5484] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 reported cstate change: term changed from 1 to 2, leader changed from bb06a516d33b4e5eb4b92499c564419c (127.26.25.129) to a3f8078f229c410e84253b2dcd90cee9 (127.26.25.133). New cstate: current_term: 2 leader_uuid: "a3f8078f229c410e84253b2dcd90cee9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:43.213783 27306 raft_consensus.cc:1261] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 2 FOLLOWER]: Refusing update from remote peer a3f8078f229c410e84253b2dcd90cee9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 13:59:43.214282 27880 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 13:59:43.217311 26912 raft_consensus.cc:1261] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 2 FOLLOWER]: Refusing update from remote peer a3f8078f229c410e84253b2dcd90cee9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 13:59:43.217994 27880 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 13:59:43.221896 27374 consensus_peers.cc:485] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 -> Peer e02f75b670d241e6b22c2e23c806944d (127.26.25.130:35879): Couldn't send request to peer e02f75b670d241e6b22c2e23c806944d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d207d27d13e54bdda9946a91d116248c. This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:43.673949 27900 ts_tablet_manager.cc:902] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d: Initiating tablet copy from peer a3f8078f229c410e84253b2dcd90cee9 (127.26.25.133:42515)
I20240502 13:59:43.674686 27900 tablet_copy_client.cc:308] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:42515
I20240502 13:59:43.678604 27460 tablet_copy_service.cc:140] P a3f8078f229c410e84253b2dcd90cee9: Received BeginTabletCopySession request for tablet d207d27d13e54bdda9946a91d116248c from peer e02f75b670d241e6b22c2e23c806944d ({username='slave'} at 127.26.25.130:34889)
I20240502 13:59:43.678735 27460 tablet_copy_service.cc:161] P a3f8078f229c410e84253b2dcd90cee9: Beginning new tablet copy session on tablet d207d27d13e54bdda9946a91d116248c from peer e02f75b670d241e6b22c2e23c806944d at {username='slave'} at 127.26.25.130:34889: session id = e02f75b670d241e6b22c2e23c806944d-d207d27d13e54bdda9946a91d116248c
I20240502 13:59:43.680308 27460 tablet_copy_source_session.cc:205] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 13:59:43.683387 27900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d207d27d13e54bdda9946a91d116248c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:43.688392 27900 tablet_copy_client.cc:783] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d: tablet copy: Starting download of 0 data blocks...
I20240502 13:59:43.688598 27900 tablet_copy_client.cc:647] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d: tablet copy: Starting download of 1 WAL segments...
I20240502 13:59:43.690292 27900 tablet_copy_client.cc:523] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 13:59:43.691716 27900 tablet_bootstrap.cc:492] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d: Bootstrap starting.
I20240502 13:59:43.696149 27900 log.cc:824] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:43.698623 27900 tablet_bootstrap.cc:492] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:43.698879 27900 tablet_bootstrap.cc:492] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d: Bootstrap complete.
I20240502 13:59:43.699132 27900 ts_tablet_manager.cc:1367] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20240502 13:59:43.701516 27900 raft_consensus.cc:348] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: true } }
I20240502 13:59:43.701721 27900 raft_consensus.cc:729] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e02f75b670d241e6b22c2e23c806944d, State: Initialized, Role: LEARNER
I20240502 13:59:43.702049 27900 consensus_queue.cc:260] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: true } }
I20240502 13:59:43.702879 27900 ts_tablet_manager.cc:1398] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20240502 13:59:43.703665 27460 tablet_copy_service.cc:342] P a3f8078f229c410e84253b2dcd90cee9: Request end of tablet copy session e02f75b670d241e6b22c2e23c806944d-d207d27d13e54bdda9946a91d116248c received from {username='slave'} at 127.26.25.130:34889
I20240502 13:59:43.703770 27460 tablet_copy_service.cc:433] P a3f8078f229c410e84253b2dcd90cee9: ending tablet copy session e02f75b670d241e6b22c2e23c806944d-d207d27d13e54bdda9946a91d116248c on tablet d207d27d13e54bdda9946a91d116248c with peer e02f75b670d241e6b22c2e23c806944d
I20240502 13:59:44.225277 27043 raft_consensus.cc:1203] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d [term 2 LEARNER]: Deduplicated request from leader. Original: 1.3->[2.4-2.4] Dedup: 2.4->[]
I20240502 13:59:44.834614 27944 raft_consensus.cc:1050] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9: attempting to promote NON_VOTER e02f75b670d241e6b22c2e23c806944d to VOTER
I20240502 13:59:44.835306 27944 consensus_queue.cc:237] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false } }
I20240502 13:59:44.836782 27043 raft_consensus.cc:1261] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d [term 2 LEARNER]: Refusing update from remote peer a3f8078f229c410e84253b2dcd90cee9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 13:59:44.836805 27306 raft_consensus.cc:1261] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 2 FOLLOWER]: Refusing update from remote peer a3f8078f229c410e84253b2dcd90cee9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 13:59:44.836843 26912 raft_consensus.cc:1261] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 2 FOLLOWER]: Refusing update from remote peer a3f8078f229c410e84253b2dcd90cee9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 13:59:44.837255 27947 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:44.837623 27944 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:44.837821 27927 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:44.840585 27927 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, e02f75b670d241e6b22c2e23c806944d (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false } } }
I20240502 13:59:44.841162 27306 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, e02f75b670d241e6b22c2e23c806944d (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false } } }
I20240502 13:59:44.841183 27043 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, e02f75b670d241e6b22c2e23c806944d (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false } } }
I20240502 13:59:44.842835 26912 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, e02f75b670d241e6b22c2e23c806944d (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false } } }
I20240502 13:59:44.843344 26767 catalog_manager.cc:5484] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 reported cstate change: config changed from index 3 to 5, e02f75b670d241e6b22c2e23c806944d (127.26.25.130) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a3f8078f229c410e84253b2dcd90cee9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb06a516d33b4e5eb4b92499c564419c" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 39611 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:44.847087 27440 consensus_queue.cc:237] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false } }
I20240502 13:59:44.848174 27043 raft_consensus.cc:1261] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d [term 2 FOLLOWER]: Refusing update from remote peer a3f8078f229c410e84253b2dcd90cee9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 13:59:44.848269 27306 raft_consensus.cc:1261] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 2 FOLLOWER]: Refusing update from remote peer a3f8078f229c410e84253b2dcd90cee9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 13:59:44.848546 27947 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 13:59:44.848739 27944 consensus_queue.cc:1035] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 13:59:44.850111 27947 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 5 to 6, VOTER bb06a516d33b4e5eb4b92499c564419c (127.26.25.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false } } }
I20240502 13:59:44.850615 27043 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 5 to 6, VOTER bb06a516d33b4e5eb4b92499c564419c (127.26.25.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false } } }
I20240502 13:59:44.851855 26754 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d207d27d13e54bdda9946a91d116248c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 13:59:44.851979 27306 raft_consensus.cc:2932] T d207d27d13e54bdda9946a91d116248c P 758da1dffcd84542b500fc457aa067c4 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 5 to 6, VOTER bb06a516d33b4e5eb4b92499c564419c (127.26.25.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false } } }
I20240502 13:59:44.852679 26768 catalog_manager.cc:5484] T d207d27d13e54bdda9946a91d116248c P e02f75b670d241e6b22c2e23c806944d reported cstate change: config changed from index 5 to 6, VOTER bb06a516d33b4e5eb4b92499c564419c (127.26.25.129) evicted. New cstate: current_term: 2 leader_uuid: "a3f8078f229c410e84253b2dcd90cee9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a3f8078f229c410e84253b2dcd90cee9" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 42515 } } peers { permanent_uuid: "758da1dffcd84542b500fc457aa067c4" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 41709 } } peers { permanent_uuid: "e02f75b670d241e6b22c2e23c806944d" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35879 } attrs { promote: false } } }
I20240502 13:59:44.855171 26891 tablet_service.cc:1514] Processing DeleteTablet for tablet d207d27d13e54bdda9946a91d116248c with delete_type TABLET_DATA_TOMBSTONED (TS bb06a516d33b4e5eb4b92499c564419c not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:36620
I20240502 13:59:44.855645 27958 tablet_replica.cc:330] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: stopping tablet replica
I20240502 13:59:44.855985 27958 raft_consensus.cc:2229] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 13:59:44.856196 27958 raft_consensus.cc:2258] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 13:59:44.857133 27958 ts_tablet_manager.cc:1875] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 13:59:44.858770 27958 ts_tablet_manager.cc:1888] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 13:59:44.858842 27958 log.cc:1196] T d207d27d13e54bdda9946a91d116248c P bb06a516d33b4e5eb4b92499c564419c: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceIgnoredTserversTest.Basic.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/d207d27d13e54bdda9946a91d116248c
I20240502 13:59:44.859350 26756 catalog_manager.cc:4830] TS bb06a516d33b4e5eb4b92499c564419c (127.26.25.129:39611): tablet d207d27d13e54bdda9946a91d116248c (table TestTable [id=931842289a5744a0abe15a551c84a8cb]) successfully deleted
I20240502 13:59:45.176009 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.176023 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.176599 27420 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:45.184541 26891 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.185891 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.260486 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.260524 26891 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.260901 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.262472 27420 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:45.268882 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.336817 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.338544 26891 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.338830 27420 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:45.339665 27023 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.340454 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.379087 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 26961
I20240502 13:59:45.561021 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.561223 27420 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:45.562986 26891 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.569113 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.631917 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.632723 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.633375 26891 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.640308 27420 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:45.731631 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.734661 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.735286 27420 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:45.735867 26891 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240502 13:59:45.792342 27374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.25.130:35879: connect: Connection refused (error 111)
W20240502 13:59:45.793349 27374 consensus_peers.cc:485] T d207d27d13e54bdda9946a91d116248c P a3f8078f229c410e84253b2dcd90cee9 -> Peer e02f75b670d241e6b22c2e23c806944d (127.26.25.130:35879): Couldn't send request to peer e02f75b670d241e6b22c2e23c806944d. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.130:35879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:45.806301 27286 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.807986 26891 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.808220 27420 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:45.808904 27155 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:45.840792 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 26829
I20240502 13:59:45.847102 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 27093
I20240502 13:59:45.852434 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 27224
I20240502 13:59:45.858109 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 27357
W20240502 13:59:45.863600 26753 connection.cc:535] server connection from 127.26.25.133:42765 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 13:59:45.863943 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 26738
2024-05-02T13:59:45Z chronyd exiting
[ OK ] RebalanceIgnoredTserversTest.Basic (6724 ms)
[----------] 1 test from RebalanceIgnoredTserversTest (6724 ms total)
[----------] 1 test from RebalanceStartCriteriaTest
[ RUN ] RebalanceStartCriteriaTest.TabletServerIsDown/1
I20240502 13:59:45.886510 26726 test_util.cc:268] Using random seed: 67140906
I20240502 13:59:45.887220 26726 ts_itest-base.cc:112] Starting cluster with:
I20240502 13:59:45.887261 26726 ts_itest-base.cc:113] --------------
I20240502 13:59:45.887272 26726 ts_itest-base.cc:114] 5 tablet servers
I20240502 13:59:45.887284 26726 ts_itest-base.cc:115] 3 replicas per TS
I20240502 13:59:45.887297 26726 ts_itest-base.cc:116] --------------
2024-05-02T13:59:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T13:59:45Z Disabled control of system clock
I20240502 13:59:45.894791 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.25.190:42115
--webserver_interface=127.26.25.190
--webserver_port=0
--builtin_ntp_servers=127.26.25.148:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.25.190:42115
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240502 13:59:45.998484 28181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:45.998693 28181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:45.998762 28181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:46.001688 28181 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:46.001761 28181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 13:59:46.001796 28181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:46.001839 28181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 13:59:46.001883 28181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 13:59:46.005383 28181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.26.25.190:42115
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.25.190:42115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.26.25.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.28181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:46.006217 28181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:46.007272 28181 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240502 13:59:46.015163 28181 server_base.cc:1034] running on GCE node
W20240502 13:59:46.015798 28187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.015951 28189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.016338 28186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:46.016884 28181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:46.017736 28181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:46.018923 28181 hybrid_clock.cc:648] HybridClock initialized: now 1714658386018916 us; error 27 us; skew 500 ppm
I20240502 13:59:46.020753 28181 webserver.cc:460] Webserver started at http://127.26.25.190:44267/ using document root <none> and password file <none>
I20240502 13:59:46.021277 28181 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:46.021339 28181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:46.021507 28181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:46.023042 28181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "af514af1b0af484cad4885f22d375759"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.023483 28181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "af514af1b0af484cad4885f22d375759"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.026508 28181 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:46.028484 28196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.029467 28181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:46.029574 28181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
uuid: "af514af1b0af484cad4885f22d375759"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.029659 28181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:46.054949 28181 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:46.055577 28181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:46.055735 28181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:46.062108 28181 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.190:42115
I20240502 13:59:46.062119 28248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.190:42115 every 8 connection(s)
I20240502 13:59:46.063061 28181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 13:59:46.065502 28249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:46.069514 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 28181
I20240502 13:59:46.069610 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 13:59:46.070142 28249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759: Bootstrap starting.
I20240502 13:59:46.072297 28249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:46.072917 28249 log.cc:824] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:46.074342 28249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759: No bootstrap required, opened a new log
I20240502 13:59:46.077035 28249 raft_consensus.cc:348] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af514af1b0af484cad4885f22d375759" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42115 } }
I20240502 13:59:46.077231 28249 raft_consensus.cc:374] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:46.077287 28249 raft_consensus.cc:729] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af514af1b0af484cad4885f22d375759, State: Initialized, Role: FOLLOWER
I20240502 13:59:46.077718 28249 consensus_queue.cc:260] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af514af1b0af484cad4885f22d375759" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42115 } }
I20240502 13:59:46.077860 28249 raft_consensus.cc:388] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:46.077919 28249 raft_consensus.cc:482] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:46.078023 28249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:46.078804 28249 raft_consensus.cc:504] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af514af1b0af484cad4885f22d375759" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42115 } }
I20240502 13:59:46.079077 28249 leader_election.cc:302] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af514af1b0af484cad4885f22d375759; no voters:
I20240502 13:59:46.079290 28249 leader_election.cc:288] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:46.079375 28254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:46.079596 28254 raft_consensus.cc:686] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [term 1 LEADER]: Becoming Leader. State: Replica: af514af1b0af484cad4885f22d375759, State: Running, Role: LEADER
I20240502 13:59:46.079936 28254 consensus_queue.cc:237] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af514af1b0af484cad4885f22d375759" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42115 } }
I20240502 13:59:46.080265 28249 sys_catalog.cc:564] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 13:59:46.081387 28255 sys_catalog.cc:455] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af514af1b0af484cad4885f22d375759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af514af1b0af484cad4885f22d375759" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42115 } } }
I20240502 13:59:46.081388 28256 sys_catalog.cc:455] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af514af1b0af484cad4885f22d375759. Latest consensus state: current_term: 1 leader_uuid: "af514af1b0af484cad4885f22d375759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af514af1b0af484cad4885f22d375759" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42115 } } }
I20240502 13:59:46.081555 28255 sys_catalog.cc:458] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:46.081705 28256 sys_catalog.cc:458] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759 [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:46.082353 28263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 13:59:46.085055 28263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 13:59:46.089658 28263 catalog_manager.cc:1347] Generated new cluster ID: 6ad7248c78a247b4a9c33fa445b89c91
I20240502 13:59:46.089720 28263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 13:59:46.106950 28263 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 13:59:46.107790 28263 catalog_manager.cc:1504] Loading token signing keys...
I20240502 13:59:46.118388 28263 catalog_manager.cc:5857] T 00000000000000000000000000000000 P af514af1b0af484cad4885f22d375759: Generated new TSK 0
I20240502 13:59:46.119024 28263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 13:59:46.125785 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.129:0
--local_ip_for_outbound_sockets=127.26.25.129
--webserver_interface=127.26.25.129
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42115
--builtin_ntp_servers=127.26.25.148:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 13:59:46.230703 28274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:46.230903 28274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:46.230952 28274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:46.234040 28274 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:46.234104 28274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:46.234189 28274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.129
I20240502 13:59:46.237741 28274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.26.25.129
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42115
--never_fsync=true
--heap_profile_path=/tmp/kudu.28274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.129
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:46.238587 28274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:46.239668 28274 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:46.249027 28279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.249672 28282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.250453 28280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:46.252354 28274 server_base.cc:1034] running on GCE node
I20240502 13:59:46.252787 28274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:46.253396 28274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:46.254561 28274 hybrid_clock.cc:648] HybridClock initialized: now 1714658386254541 us; error 32 us; skew 500 ppm
I20240502 13:59:46.256418 28274 webserver.cc:460] Webserver started at http://127.26.25.129:43963/ using document root <none> and password file <none>
I20240502 13:59:46.256953 28274 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:46.257009 28274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:46.257218 28274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:46.258809 28274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "1e4a26d603fd4282a7e323cb93dc9275"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.259254 28274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "1e4a26d603fd4282a7e323cb93dc9275"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.262375 28274 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:46.264375 28288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.265295 28274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:46.265410 28274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "1e4a26d603fd4282a7e323cb93dc9275"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.265487 28274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:46.283259 28274 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:46.283810 28274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:46.284004 28274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:46.284490 28274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:46.285326 28274 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:46.285377 28274 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.285434 28274 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:46.285467 28274 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.293592 28274 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.129:44237
I20240502 13:59:46.293646 28402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.129:44237 every 8 connection(s)
I20240502 13:59:46.294463 28274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 13:59:46.301223 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 28274
I20240502 13:59:46.301340 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 13:59:46.302881 28403 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42115
I20240502 13:59:46.303090 28403 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:46.303376 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.130:0
--local_ip_for_outbound_sockets=127.26.25.130
--webserver_interface=127.26.25.130
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42115
--builtin_ntp_servers=127.26.25.148:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:46.303576 28403 heartbeater.cc:511] Master 127.26.25.190:42115 requested a full tablet report, sending...
I20240502 13:59:46.304970 28213 ts_manager.cc:194] Registered new tserver with Master: 1e4a26d603fd4282a7e323cb93dc9275 (127.26.25.129:44237)
I20240502 13:59:46.306069 28213 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.129:42221
W20240502 13:59:46.407784 28406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:46.407994 28406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:46.408037 28406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:46.410812 28406 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:46.410862 28406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:46.410921 28406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.130
I20240502 13:59:46.414317 28406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.26.25.130
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42115
--never_fsync=true
--heap_profile_path=/tmp/kudu.28406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.130
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:46.415091 28406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:46.416121 28406 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:46.423349 28412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.423928 28414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.424399 28411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:46.424932 28406 server_base.cc:1034] running on GCE node
I20240502 13:59:46.425216 28406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:46.425668 28406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:46.426810 28406 hybrid_clock.cc:648] HybridClock initialized: now 1714658386426782 us; error 39 us; skew 500 ppm
I20240502 13:59:46.428498 28406 webserver.cc:460] Webserver started at http://127.26.25.130:33841/ using document root <none> and password file <none>
I20240502 13:59:46.428992 28406 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:46.429056 28406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:46.429221 28406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:46.430804 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "5314418d4646487286d2a8db02ad701a"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.431258 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "5314418d4646487286d2a8db02ad701a"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.434422 28406 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:46.436367 28420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.437237 28406 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:46.437325 28406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "5314418d4646487286d2a8db02ad701a"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.437408 28406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:46.458822 28406 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:46.459321 28406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:46.459467 28406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:46.459986 28406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:46.460757 28406 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:46.460800 28406 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.460844 28406 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:46.460875 28406 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.468392 28406 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.130:32787
I20240502 13:59:46.468473 28534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.130:32787 every 8 connection(s)
I20240502 13:59:46.469230 28406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 13:59:46.470111 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 28406
I20240502 13:59:46.470206 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 13:59:46.472220 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.131:0
--local_ip_for_outbound_sockets=127.26.25.131
--webserver_interface=127.26.25.131
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42115
--builtin_ntp_servers=127.26.25.148:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:46.477674 28535 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42115
I20240502 13:59:46.477869 28535 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:46.478318 28535 heartbeater.cc:511] Master 127.26.25.190:42115 requested a full tablet report, sending...
I20240502 13:59:46.479240 28213 ts_manager.cc:194] Registered new tserver with Master: 5314418d4646487286d2a8db02ad701a (127.26.25.130:32787)
I20240502 13:59:46.479772 28213 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.130:57763
W20240502 13:59:46.573119 28538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:46.573308 28538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:46.573351 28538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:46.576382 28538 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:46.576442 28538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:46.576537 28538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.131
I20240502 13:59:46.580081 28538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.26.25.131
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42115
--never_fsync=true
--heap_profile_path=/tmp/kudu.28538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.131
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:46.580886 28538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:46.581990 28538 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:46.589864 28544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.590447 28543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.591075 28546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:46.591735 28538 server_base.cc:1034] running on GCE node
I20240502 13:59:46.592124 28538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:46.592626 28538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:46.593784 28538 hybrid_clock.cc:648] HybridClock initialized: now 1714658386593777 us; error 27 us; skew 500 ppm
I20240502 13:59:46.595525 28538 webserver.cc:460] Webserver started at http://127.26.25.131:34349/ using document root <none> and password file <none>
I20240502 13:59:46.596089 28538 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:46.596151 28538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:46.596314 28538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:46.597975 28538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "418bb40cda504c3685232800fc2d5919"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.598487 28538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "418bb40cda504c3685232800fc2d5919"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.601871 28538 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:46.603982 28552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.604894 28538 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:46.605010 28538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "418bb40cda504c3685232800fc2d5919"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.605083 28538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:46.643994 28538 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:46.644644 28538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:46.644824 28538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:46.645342 28538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:46.646277 28538 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:46.646342 28538 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.646417 28538 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:46.646446 28538 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.655413 28538 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.131:33923
I20240502 13:59:46.655447 28666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.131:33923 every 8 connection(s)
I20240502 13:59:46.656359 28538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 13:59:46.658308 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 28538
I20240502 13:59:46.658407 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 13:59:46.660527 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.132:0
--local_ip_for_outbound_sockets=127.26.25.132
--webserver_interface=127.26.25.132
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42115
--builtin_ntp_servers=127.26.25.148:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:46.665494 28667 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42115
I20240502 13:59:46.665706 28667 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:46.666188 28667 heartbeater.cc:511] Master 127.26.25.190:42115 requested a full tablet report, sending...
I20240502 13:59:46.667073 28213 ts_manager.cc:194] Registered new tserver with Master: 418bb40cda504c3685232800fc2d5919 (127.26.25.131:33923)
I20240502 13:59:46.667609 28213 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.131:33113
W20240502 13:59:46.766563 28670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:46.766778 28670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:46.766822 28670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:46.769937 28670 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:46.770001 28670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:46.770094 28670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.132
I20240502 13:59:46.773654 28670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.26.25.132
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42115
--never_fsync=true
--heap_profile_path=/tmp/kudu.28670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.132
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:46.774513 28670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:46.775625 28670 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:46.783903 28675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.784333 28678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.784818 28676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:46.785653 28670 server_base.cc:1034] running on GCE node
I20240502 13:59:46.786028 28670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:46.786589 28670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:46.787737 28670 hybrid_clock.cc:648] HybridClock initialized: now 1714658386787712 us; error 39 us; skew 500 ppm
I20240502 13:59:46.789552 28670 webserver.cc:460] Webserver started at http://127.26.25.132:41343/ using document root <none> and password file <none>
I20240502 13:59:46.790061 28670 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:46.790113 28670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:46.790269 28670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:46.791867 28670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "622d8ac546f749119ed26531767d8566"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.792313 28670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "622d8ac546f749119ed26531767d8566"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.795403 28670 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 13:59:46.797514 28684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.798575 28670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240502 13:59:46.798668 28670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "622d8ac546f749119ed26531767d8566"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.798754 28670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:46.809518 28670 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:46.810046 28670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:46.810194 28670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:46.810678 28670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:46.811517 28670 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:46.811563 28670 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.811606 28670 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:46.811640 28670 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.820878 28670 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.132:33787
I20240502 13:59:46.820914 28798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.132:33787 every 8 connection(s)
I20240502 13:59:46.821887 28670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 13:59:46.826007 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 28670
I20240502 13:59:46.826129 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 13:59:46.828444 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.133:0
--local_ip_for_outbound_sockets=127.26.25.133
--webserver_interface=127.26.25.133
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42115
--builtin_ntp_servers=127.26.25.148:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:46.830840 28799 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42115
I20240502 13:59:46.831045 28799 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:46.831555 28799 heartbeater.cc:511] Master 127.26.25.190:42115 requested a full tablet report, sending...
I20240502 13:59:46.832444 28213 ts_manager.cc:194] Registered new tserver with Master: 622d8ac546f749119ed26531767d8566 (127.26.25.132:33787)
I20240502 13:59:46.832991 28213 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.132:40955
W20240502 13:59:46.938486 28802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:46.938730 28802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:46.938781 28802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:46.942216 28802 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:46.942293 28802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:46.942409 28802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.133
I20240502 13:59:46.946218 28802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.25.133
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42115
--never_fsync=true
--heap_profile_path=/tmp/kudu.28802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.133
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:46.947091 28802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:46.948277 28802 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:46.956447 28807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.956825 28808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.957319 28810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:46.958128 28802 server_base.cc:1034] running on GCE node
I20240502 13:59:46.958460 28802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:46.958972 28802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:46.960115 28802 hybrid_clock.cc:648] HybridClock initialized: now 1714658386960106 us; error 27 us; skew 500 ppm
I20240502 13:59:46.961920 28802 webserver.cc:460] Webserver started at http://127.26.25.133:40011/ using document root <none> and password file <none>
I20240502 13:59:46.962436 28802 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:46.962504 28802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:46.962693 28802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:46.964491 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "2a749a1b944f4de2b286a126295477e6"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.964953 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "2a749a1b944f4de2b286a126295477e6"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.968109 28802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 13:59:46.970219 28816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.971138 28802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:46.971240 28802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "2a749a1b944f4de2b286a126295477e6"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-sldb"
I20240502 13:59:46.971316 28802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:46.991012 28802 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:46.991570 28802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:46.991722 28802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:46.992285 28802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:46.993139 28802 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:46.993191 28802 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.993244 28802 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:46.993273 28802 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:47.001431 28802 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.133:46759
I20240502 13:59:47.001451 28929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.133:46759 every 8 connection(s)
I20240502 13:59:47.002339 28802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 13:59:47.005151 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 28802
I20240502 13:59:47.005261 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceStartCriteriaTest.TabletServerIsDown_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 13:59:47.010882 28930 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42115
I20240502 13:59:47.011082 28930 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:47.011562 28930 heartbeater.cc:511] Master 127.26.25.190:42115 requested a full tablet report, sending...
I20240502 13:59:47.012501 28213 ts_manager.cc:194] Registered new tserver with Master: 2a749a1b944f4de2b286a126295477e6 (127.26.25.133:46759)
I20240502 13:59:47.013069 28213 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.133:52499
I20240502 13:59:47.019003 26726 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 13:59:47.027779 28213 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35002:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 13:59:47.045658 28732 tablet_service.cc:1467] Processing CreateTablet for tablet 9fbb02f1f1f343f3a858f68f7667cb31 (DEFAULT_TABLE table=TestTable [id=053d2b561f7b46049e7c972229418f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:47.045660 28469 tablet_service.cc:1467] Processing CreateTablet for tablet 9fbb02f1f1f343f3a858f68f7667cb31 (DEFAULT_TABLE table=TestTable [id=053d2b561f7b46049e7c972229418f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:47.045635 28600 tablet_service.cc:1467] Processing CreateTablet for tablet 9fbb02f1f1f343f3a858f68f7667cb31 (DEFAULT_TABLE table=TestTable [id=053d2b561f7b46049e7c972229418f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:47.046579 28600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fbb02f1f1f343f3a858f68f7667cb31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:47.046582 28469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fbb02f1f1f343f3a858f68f7667cb31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:47.046582 28732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fbb02f1f1f343f3a858f68f7667cb31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:47.051576 28947 tablet_bootstrap.cc:492] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919: Bootstrap starting.
I20240502 13:59:47.052701 28948 tablet_bootstrap.cc:492] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a: Bootstrap starting.
I20240502 13:59:47.053395 28949 tablet_bootstrap.cc:492] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566: Bootstrap starting.
I20240502 13:59:47.054188 28947 tablet_bootstrap.cc:654] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:47.054325 28948 tablet_bootstrap.cc:654] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:47.054940 28948 log.cc:824] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:47.054966 28947 log.cc:824] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:47.055284 28949 tablet_bootstrap.cc:654] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:47.055958 28949 log.cc:824] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:47.056664 28948 tablet_bootstrap.cc:492] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a: No bootstrap required, opened a new log
I20240502 13:59:47.056828 28948 ts_tablet_manager.cc:1367] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 13:59:47.057476 28949 tablet_bootstrap.cc:492] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566: No bootstrap required, opened a new log
I20240502 13:59:47.057636 28949 ts_tablet_manager.cc:1367] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 13:59:47.060602 28948 raft_consensus.cc:348] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622d8ac546f749119ed26531767d8566" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33787 } } peers { permanent_uuid: "5314418d4646487286d2a8db02ad701a" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 32787 } } peers { permanent_uuid: "418bb40cda504c3685232800fc2d5919" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 33923 } }
I20240502 13:59:47.060830 28948 raft_consensus.cc:374] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:47.060876 28948 raft_consensus.cc:729] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5314418d4646487286d2a8db02ad701a, State: Initialized, Role: FOLLOWER
I20240502 13:59:47.061331 28948 consensus_queue.cc:260] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622d8ac546f749119ed26531767d8566" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33787 } } peers { permanent_uuid: "5314418d4646487286d2a8db02ad701a" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 32787 } } peers { permanent_uuid: "418bb40cda504c3685232800fc2d5919" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 33923 } }
I20240502 13:59:47.061338 28949 raft_consensus.cc:348] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622d8ac546f749119ed26531767d8566" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33787 } } peers { permanent_uuid: "5314418d4646487286d2a8db02ad701a" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 32787 } } peers { permanent_uuid: "418bb40cda504c3685232800fc2d5919" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 33923 } }
I20240502 13:59:47.061502 28949 raft_consensus.cc:374] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:47.061578 28949 raft_consensus.cc:729] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622d8ac546f749119ed26531767d8566, State: Initialized, Role: FOLLOWER
I20240502 13:59:47.061925 28949 consensus_queue.cc:260] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622d8ac546f749119ed26531767d8566" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33787 } } peers { permanent_uuid: "5314418d4646487286d2a8db02ad701a" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 32787 } } peers { permanent_uuid: "418bb40cda504c3685232800fc2d5919" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 33923 } }
I20240502 13:59:47.062204 28948 ts_tablet_manager.cc:1398] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240502 13:59:47.062566 28799 heartbeater.cc:503] Master 127.26.25.190:42115 was elected leader, sending a full tablet report...
I20240502 13:59:47.062564 28535 heartbeater.cc:503] Master 127.26.25.190:42115 was elected leader, sending a full tablet report...
I20240502 13:59:47.063694 28947 tablet_bootstrap.cc:492] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919: No bootstrap required, opened a new log
I20240502 13:59:47.063809 28949 ts_tablet_manager.cc:1398] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240502 13:59:47.063907 28947 ts_tablet_manager.cc:1367] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20240502 13:59:47.066658 28947 raft_consensus.cc:348] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622d8ac546f749119ed26531767d8566" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33787 } } peers { permanent_uuid: "5314418d4646487286d2a8db02ad701a" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 32787 } } peers { permanent_uuid: "418bb40cda504c3685232800fc2d5919" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 33923 } }
I20240502 13:59:47.066807 28947 raft_consensus.cc:374] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:47.066836 28947 raft_consensus.cc:729] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418bb40cda504c3685232800fc2d5919, State: Initialized, Role: FOLLOWER
I20240502 13:59:47.067181 28947 consensus_queue.cc:260] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622d8ac546f749119ed26531767d8566" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33787 } } peers { permanent_uuid: "5314418d4646487286d2a8db02ad701a" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 32787 } } peers { permanent_uuid: "418bb40cda504c3685232800fc2d5919" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 33923 } }
I20240502 13:59:47.067783 28667 heartbeater.cc:503] Master 127.26.25.190:42115 was elected leader, sending a full tablet report...
I20240502 13:59:47.067940 28947 ts_tablet_manager.cc:1398] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20240502 13:59:47.072705 28800 tablet.cc:2369] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 13:59:47.157438 28668 tablet.cc:2369] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:47.169711 28955 raft_consensus.cc:482] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:47.169886 28955 raft_consensus.cc:504] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622d8ac546f749119ed26531767d8566" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33787 } } peers { permanent_uuid: "5314418d4646487286d2a8db02ad701a" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 32787 } } peers { permanent_uuid: "418bb40cda504c3685232800fc2d5919" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 33923 } }
I20240502 13:59:47.170881 28955 leader_election.cc:288] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 622d8ac546f749119ed26531767d8566 (127.26.25.132:33787), 5314418d4646487286d2a8db02ad701a (127.26.25.130:32787)
I20240502 13:59:47.174544 28752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fbb02f1f1f343f3a858f68f7667cb31" candidate_uuid: "418bb40cda504c3685232800fc2d5919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622d8ac546f749119ed26531767d8566" is_pre_election: true
I20240502 13:59:47.174566 28489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fbb02f1f1f343f3a858f68f7667cb31" candidate_uuid: "418bb40cda504c3685232800fc2d5919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5314418d4646487286d2a8db02ad701a" is_pre_election: true
I20240502 13:59:47.174805 28489 raft_consensus.cc:2454] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 418bb40cda504c3685232800fc2d5919 in term 0.
I20240502 13:59:47.174823 28752 raft_consensus.cc:2454] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 418bb40cda504c3685232800fc2d5919 in term 0.
I20240502 13:59:47.175273 28556 leader_election.cc:302] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 418bb40cda504c3685232800fc2d5919, 622d8ac546f749119ed26531767d8566; no voters:
I20240502 13:59:47.175552 28955 raft_consensus.cc:2781] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:47.175621 28955 raft_consensus.cc:482] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:47.175650 28955 raft_consensus.cc:3037] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:47.219913 28955 raft_consensus.cc:504] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622d8ac546f749119ed26531767d8566" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33787 } } peers { permanent_uuid: "5314418d4646487286d2a8db02ad701a" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 32787 } } peers { permanent_uuid: "418bb40cda504c3685232800fc2d5919" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 33923 } }
W20240502 13:59:47.220465 28536 tablet.cc:2369] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:47.220928 28752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fbb02f1f1f343f3a858f68f7667cb31" candidate_uuid: "418bb40cda504c3685232800fc2d5919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622d8ac546f749119ed26531767d8566"
I20240502 13:59:47.221093 28955 leader_election.cc:288] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [CANDIDATE]: Term 1 election: Requested vote from peers 622d8ac546f749119ed26531767d8566 (127.26.25.132:33787), 5314418d4646487286d2a8db02ad701a (127.26.25.130:32787)
I20240502 13:59:47.221071 28752 raft_consensus.cc:3037] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:47.221263 28489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fbb02f1f1f343f3a858f68f7667cb31" candidate_uuid: "418bb40cda504c3685232800fc2d5919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5314418d4646487286d2a8db02ad701a"
I20240502 13:59:47.221428 28489 raft_consensus.cc:3037] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:47.222259 28752 raft_consensus.cc:2454] T 9fbb02f1f1f343f3a858f68f7667cb31 P 622d8ac546f749119ed26531767d8566 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 418bb40cda504c3685232800fc2d5919 in term 1.
I20240502 13:59:47.222617 28556 leader_election.cc:302] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 418bb40cda504c3685232800fc2d5919, 622d8ac546f749119ed26531767d8566; no voters:
I20240502 13:59:47.222807 28955 raft_consensus.cc:2781] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:47.222826 28489 raft_consensus.cc:2454] T 9fbb02f1f1f343f3a858f68f7667cb31 P 5314418d4646487286d2a8db02ad701a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 418bb40cda504c3685232800fc2d5919 in term 1.
I20240502 13:59:47.223038 28955 raft_consensus.cc:686] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [term 1 LEADER]: Becoming Leader. State: Replica: 418bb40cda504c3685232800fc2d5919, State: Running, Role: LEADER
I20240502 13:59:47.223526 28955 consensus_queue.cc:237] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622d8ac546f749119ed26531767d8566" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33787 } } peers { permanent_uuid: "5314418d4646487286d2a8db02ad701a" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 32787 } } peers { permanent_uuid: "418bb40cda504c3685232800fc2d5919" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 33923 } }
I20240502 13:59:47.226594 28213 catalog_manager.cc:5484] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 418bb40cda504c3685232800fc2d5919 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "418bb40cda504c3685232800fc2d5919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622d8ac546f749119ed26531767d8566" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5314418d4646487286d2a8db02ad701a" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 32787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "418bb40cda504c3685232800fc2d5919" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 33923 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:47.236670 26726 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 13:59:47.238078 26726 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 1e4a26d603fd4282a7e323cb93dc9275 to finish bootstrapping
I20240502 13:59:47.244345 26726 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 5314418d4646487286d2a8db02ad701a to finish bootstrapping
I20240502 13:59:47.249537 26726 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 418bb40cda504c3685232800fc2d5919 to finish bootstrapping
I20240502 13:59:47.254611 26726 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 622d8ac546f749119ed26531767d8566 to finish bootstrapping
I20240502 13:59:47.259531 26726 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 2a749a1b944f4de2b286a126295477e6 to finish bootstrapping
I20240502 13:59:47.265211 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 28274
I20240502 13:59:47.455410 28864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.456594 28469 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.456701 28600 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:47.457859 28732 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.532807 28864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.534904 28732 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.535491 28600 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:47.535988 28469 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.642462 28955 consensus_queue.cc:1035] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "622d8ac546f749119ed26531767d8566" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:47.644834 28600 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:47.645371 28469 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.647174 28864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.651127 28958 consensus_queue.cc:1035] T 9fbb02f1f1f343f3a858f68f7667cb31 P 418bb40cda504c3685232800fc2d5919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5314418d4646487286d2a8db02ad701a" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 32787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:47.658484 28732 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.867427 28864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.867499 28469 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.868428 28600 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:47.868610 28732 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.941313 28600 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:47.942956 28469 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.943326 28864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.948732 28732 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.014768 28930 heartbeater.cc:503] Master 127.26.25.190:42115 was elected leader, sending a full tablet report...
I20240502 13:59:48.049525 28864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.049707 28469 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.050119 28600 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:48.051496 28732 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.088647 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 28406
I20240502 13:59:48.095054 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 28538
I20240502 13:59:48.100721 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 28670
I20240502 13:59:48.106184 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 28802
I20240502 13:59:48.111934 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 28181
2024-05-02T13:59:48Z chronyd exiting
[ OK ] RebalanceStartCriteriaTest.TabletServerIsDown/1 (2248 ms)
[----------] 1 test from RebalanceStartCriteriaTest (2248 ms total)
[----------] 1 test from RebalanceParamTest
[ RUN ] RebalanceParamTest.Rebalance/1
I20240502 13:59:48.134738 26726 test_util.cc:268] Using random seed: 69389134
I20240502 13:59:48.135483 26726 ts_itest-base.cc:112] Starting cluster with:
I20240502 13:59:48.135519 26726 ts_itest-base.cc:113] --------------
I20240502 13:59:48.135530 26726 ts_itest-base.cc:114] 7 tablet servers
I20240502 13:59:48.135540 26726 ts_itest-base.cc:115] 1 replicas per TS
I20240502 13:59:48.135564 26726 ts_itest-base.cc:116] --------------
2024-05-02T13:59:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T13:59:48Z Disabled control of system clock
I20240502 13:59:48.143452 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.25.190:33009
--webserver_interface=127.26.25.190
--webserver_port=0
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.25.190:33009
--allow_unsafe_replication_factor
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=3000 with env {}
W20240502 13:59:48.248786 29157 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240502 13:59:48.249063 29157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:48.249102 29157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:48.249146 29157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:48.252097 29157 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:48.252161 29157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 13:59:48.252184 29157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:48.252202 29157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 13:59:48.252219 29157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 13:59:48.255766 29157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.26.25.190:33009
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.25.190:33009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.26.25.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.29157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:48.256641 29157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:48.257727 29157 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:48.265183 29162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.265868 29163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.266539 29165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:48.267226 29157 server_base.cc:1034] running on GCE node
I20240502 13:59:48.267776 29157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:48.268733 29157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:48.269884 29157 hybrid_clock.cc:648] HybridClock initialized: now 1714658388269869 us; error 42 us; skew 500 ppm
I20240502 13:59:48.271699 29157 webserver.cc:460] Webserver started at http://127.26.25.190:38847/ using document root <none> and password file <none>
I20240502 13:59:48.272248 29157 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:48.272306 29157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:48.272492 29157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:48.274152 29157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "dc2396ac20b549299f014b770066f0c8"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.274632 29157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "dc2396ac20b549299f014b770066f0c8"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.277894 29157 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:48.280218 29171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.281188 29157 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:48.281309 29157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
uuid: "dc2396ac20b549299f014b770066f0c8"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.281394 29157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:48.306087 29157 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:48.306746 29157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:48.306907 29157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:48.313998 29157 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.190:33009
I20240502 13:59:48.314005 29224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.190:33009 every 8 connection(s)
I20240502 13:59:48.315016 29157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 13:59:48.317585 29225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:48.318523 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 29157
I20240502 13:59:48.318642 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 13:59:48.323396 29225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8: Bootstrap starting.
I20240502 13:59:48.325701 29225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:48.326573 29225 log.cc:824] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:48.328186 29225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8: No bootstrap required, opened a new log
I20240502 13:59:48.330616 29225 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2396ac20b549299f014b770066f0c8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 33009 } }
I20240502 13:59:48.330835 29225 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:48.330925 29225 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc2396ac20b549299f014b770066f0c8, State: Initialized, Role: FOLLOWER
I20240502 13:59:48.331390 29225 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2396ac20b549299f014b770066f0c8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 33009 } }
I20240502 13:59:48.331545 29225 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:48.331617 29225 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:48.331712 29225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:48.344404 29225 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2396ac20b549299f014b770066f0c8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 33009 } }
I20240502 13:59:48.344831 29225 leader_election.cc:302] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc2396ac20b549299f014b770066f0c8; no voters:
I20240502 13:59:48.345173 29225 leader_election.cc:288] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:48.345611 29230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:48.345834 29230 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [term 1 LEADER]: Becoming Leader. State: Replica: dc2396ac20b549299f014b770066f0c8, State: Running, Role: LEADER
I20240502 13:59:48.346185 29230 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2396ac20b549299f014b770066f0c8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 33009 } }
I20240502 13:59:48.346429 29225 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 13:59:48.347972 29232 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc2396ac20b549299f014b770066f0c8. Latest consensus state: current_term: 1 leader_uuid: "dc2396ac20b549299f014b770066f0c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2396ac20b549299f014b770066f0c8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 33009 } } }
I20240502 13:59:48.348107 29232 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:48.348603 29231 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc2396ac20b549299f014b770066f0c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc2396ac20b549299f014b770066f0c8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 33009 } } }
I20240502 13:59:48.349037 29231 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8 [sys.catalog]: This master's current role is: LEADER
W20240502 13:59:48.351290 29245 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 13:59:48.351369 29245 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 13:59:48.351434 29241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 13:59:48.352207 29241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 13:59:48.356855 29241 catalog_manager.cc:1347] Generated new cluster ID: 7cf81980bdea4838a56d3fdf18a7ccee
I20240502 13:59:48.356928 29241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 13:59:48.377774 29241 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 13:59:48.378981 29241 catalog_manager.cc:1504] Loading token signing keys...
I20240502 13:59:48.390405 29241 catalog_manager.cc:5857] T 00000000000000000000000000000000 P dc2396ac20b549299f014b770066f0c8: Generated new TSK 0
I20240502 13:59:48.391171 29241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 13:59:48.407692 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.129:0
--local_ip_for_outbound_sockets=127.26.25.129
--webserver_interface=127.26.25.129
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 13:59:48.532959 29249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:48.533197 29249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:48.533244 29249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:48.536482 29249 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:48.536556 29249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:48.536639 29249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.129
I20240502 13:59:48.540287 29249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.26.25.129
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.29249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.129
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:48.541193 29249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:48.542327 29249 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:48.550808 29254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.551342 29257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.552165 29255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:48.553210 29249 server_base.cc:1034] running on GCE node
I20240502 13:59:48.553632 29249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:48.554220 29249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:48.555366 29249 hybrid_clock.cc:648] HybridClock initialized: now 1714658388555352 us; error 37 us; skew 500 ppm
I20240502 13:59:48.557269 29249 webserver.cc:460] Webserver started at http://127.26.25.129:40905/ using document root <none> and password file <none>
I20240502 13:59:48.557824 29249 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:48.557886 29249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:48.558077 29249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:48.559692 29249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.560201 29249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.563334 29249 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 13:59:48.565461 29263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.566380 29249 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240502 13:59:48.566483 29249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.566576 29249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:48.588913 29249 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:48.589591 29249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:48.589756 29249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:48.590991 29249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:48.591907 29249 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:48.591956 29249 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.592027 29249 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:48.592059 29249 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.600773 29249 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.129:35111
I20240502 13:59:48.600796 29377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.129:35111 every 8 connection(s)
I20240502 13:59:48.601677 29249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 13:59:48.604529 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 29249
I20240502 13:59:48.604641 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 13:59:48.606875 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.130:0
--local_ip_for_outbound_sockets=127.26.25.130
--webserver_interface=127.26.25.130
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:48.610945 29378 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:48.611147 29378 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:48.611625 29378 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:48.613073 29188 ts_manager.cc:194] Registered new tserver with Master: 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 13:59:48.614179 29188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.129:41863
W20240502 13:59:48.715731 29381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:48.715992 29381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:48.716070 29381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:48.719101 29381 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:48.719177 29381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:48.719272 29381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.130
I20240502 13:59:48.722865 29381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.26.25.130
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.29381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.130
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:48.723779 29381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:48.724939 29381 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:48.733777 29387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.734211 29386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.735042 29389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:48.735778 29381 server_base.cc:1034] running on GCE node
I20240502 13:59:48.736202 29381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:48.736778 29381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:48.737931 29381 hybrid_clock.cc:648] HybridClock initialized: now 1714658388737917 us; error 25 us; skew 500 ppm
I20240502 13:59:48.739756 29381 webserver.cc:460] Webserver started at http://127.26.25.130:45673/ using document root <none> and password file <none>
I20240502 13:59:48.740331 29381 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:48.740391 29381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:48.740552 29381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:48.742187 29381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.742614 29381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.745848 29381 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:48.748155 29395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.749150 29381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 13:59:48.749259 29381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.749346 29381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:48.771190 29381 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:48.771881 29381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:48.772053 29381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:48.772612 29381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:48.773520 29381 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:48.773571 29381 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.773627 29381 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:48.773659 29381 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.782097 29381 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.130:35699
I20240502 13:59:48.782135 29509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.130:35699 every 8 connection(s)
I20240502 13:59:48.783046 29381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 13:59:48.791693 29510 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:48.791952 29510 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:48.792461 29510 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:48.792932 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 29381
I20240502 13:59:48.793148 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 13:59:48.793538 29188 ts_manager.cc:194] Registered new tserver with Master: a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699)
I20240502 13:59:48.794207 29188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.130:47229
I20240502 13:59:48.795423 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.131:0
--local_ip_for_outbound_sockets=127.26.25.131
--webserver_interface=127.26.25.131
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 13:59:48.902664 29513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:48.902884 29513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:48.902925 29513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:48.905920 29513 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:48.906001 29513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:48.906093 29513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.131
I20240502 13:59:48.909631 29513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.26.25.131
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.29513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.131
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:48.910493 29513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:48.911582 29513 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:48.920271 29518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.920913 29521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.921586 29519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:48.922243 29513 server_base.cc:1034] running on GCE node
I20240502 13:59:48.922637 29513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:48.923174 29513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:48.924356 29513 hybrid_clock.cc:648] HybridClock initialized: now 1714658388924329 us; error 37 us; skew 500 ppm
I20240502 13:59:48.926245 29513 webserver.cc:460] Webserver started at http://127.26.25.131:45807/ using document root <none> and password file <none>
I20240502 13:59:48.926995 29513 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:48.927081 29513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:48.927299 29513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:48.928963 29513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "5265d55315f5413bb45818121d568e8b"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.929399 29513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "5265d55315f5413bb45818121d568e8b"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.932765 29513 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 13:59:48.934928 29527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.936038 29513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:48.936143 29513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "5265d55315f5413bb45818121d568e8b"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:48.936225 29513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:48.947769 29513 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:48.948580 29513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:48.948733 29513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:48.949357 29513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:48.950317 29513 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:48.950369 29513 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.950426 29513 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:48.950455 29513 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.959442 29513 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.131:37575
I20240502 13:59:48.959468 29641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.131:37575 every 8 connection(s)
I20240502 13:59:48.960386 29513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 13:59:48.960793 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 29513
I20240502 13:59:48.960894 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 13:59:48.962999 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.132:0
--local_ip_for_outbound_sockets=127.26.25.132
--webserver_interface=127.26.25.132
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:48.969836 29642 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:48.970105 29642 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:48.970618 29642 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:48.971647 29187 ts_manager.cc:194] Registered new tserver with Master: 5265d55315f5413bb45818121d568e8b (127.26.25.131:37575)
I20240502 13:59:48.972242 29187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.131:40277
W20240502 13:59:49.071247 29645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:49.071467 29645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:49.071518 29645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:49.074636 29645 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:49.074718 29645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:49.074844 29645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.132
I20240502 13:59:49.078506 29645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.26.25.132
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.29645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.132
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:49.079432 29645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:49.080598 29645 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:49.089118 29653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.089818 29650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.090571 29651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:49.091274 29645 server_base.cc:1034] running on GCE node
I20240502 13:59:49.091629 29645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:49.092221 29645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:49.093354 29645 hybrid_clock.cc:648] HybridClock initialized: now 1714658389093337 us; error 41 us; skew 500 ppm
I20240502 13:59:49.095196 29645 webserver.cc:460] Webserver started at http://127.26.25.132:43153/ using document root <none> and password file <none>
I20240502 13:59:49.095739 29645 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:49.095791 29645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:49.095990 29645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:49.097684 29645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "a18dcdc568884c3d9cf8dda212fc4034"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.098168 29645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "a18dcdc568884c3d9cf8dda212fc4034"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.101483 29645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:49.103636 29659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.104732 29645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:49.104825 29645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "a18dcdc568884c3d9cf8dda212fc4034"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.104899 29645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:49.135700 29645 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:49.136386 29645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:49.136529 29645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:49.137024 29645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:49.137892 29645 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:49.137939 29645 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.137977 29645 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:49.137997 29645 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.146255 29645 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.132:38647
I20240502 13:59:49.146286 29772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.132:38647 every 8 connection(s)
I20240502 13:59:49.147221 29645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 13:59:49.148921 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 29645
I20240502 13:59:49.149040 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 13:59:49.151439 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.133:0
--local_ip_for_outbound_sockets=127.26.25.133
--webserver_interface=127.26.25.133
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:49.157486 29773 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:49.157694 29773 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:49.158200 29773 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:49.159296 29187 ts_manager.cc:194] Registered new tserver with Master: a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132:38647)
I20240502 13:59:49.160169 29187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.132:47361
W20240502 13:59:49.266521 29776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:49.266798 29776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:49.266849 29776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:49.269995 29776 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:49.270067 29776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:49.270136 29776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.133
I20240502 13:59:49.273849 29776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.25.133
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.29776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.133
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:49.274762 29776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:49.275928 29776 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:49.284711 29781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.285235 29782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.285833 29784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:49.286796 29776 server_base.cc:1034] running on GCE node
I20240502 13:59:49.287222 29776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:49.287804 29776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:49.288990 29776 hybrid_clock.cc:648] HybridClock initialized: now 1714658389288967 us; error 38 us; skew 500 ppm
I20240502 13:59:49.290951 29776 webserver.cc:460] Webserver started at http://127.26.25.133:43445/ using document root <none> and password file <none>
I20240502 13:59:49.291620 29776 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:49.291690 29776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:49.292012 29776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:49.293783 29776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "0832ab8d65cc47cabeb5e0128a14c151"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.294263 29776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "0832ab8d65cc47cabeb5e0128a14c151"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.297616 29776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20240502 13:59:49.299780 29790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.300901 29776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:49.301030 29776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "0832ab8d65cc47cabeb5e0128a14c151"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.301117 29776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:49.321007 29776 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:49.321677 29776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:49.321839 29776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:49.322399 29776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:49.323310 29776 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:49.323362 29776 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.323421 29776 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:49.323450 29776 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.332448 29776 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.133:44559
I20240502 13:59:49.332481 29904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.133:44559 every 8 connection(s)
I20240502 13:59:49.333387 29776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 13:59:49.338486 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 29776
I20240502 13:59:49.338579 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 13:59:49.340988 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.134:0
--local_ip_for_outbound_sockets=127.26.25.134
--webserver_interface=127.26.25.134
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:49.342665 29905 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:49.342931 29905 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:49.343475 29905 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:49.344446 29187 ts_manager.cc:194] Registered new tserver with Master: 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133:44559)
I20240502 13:59:49.345225 29187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.133:44595
W20240502 13:59:49.449272 29908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:49.449542 29908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:49.449625 29908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:49.452698 29908 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:49.452770 29908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:49.452863 29908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.134
I20240502 13:59:49.456420 29908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.26.25.134
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.29908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.134
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:49.457254 29908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:49.458353 29908 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:49.467108 29913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.467813 29914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.468497 29916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:49.469099 29908 server_base.cc:1034] running on GCE node
I20240502 13:59:49.469475 29908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:49.469988 29908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:49.471144 29908 hybrid_clock.cc:648] HybridClock initialized: now 1714658389471130 us; error 27 us; skew 500 ppm
I20240502 13:59:49.472924 29908 webserver.cc:460] Webserver started at http://127.26.25.134:39549/ using document root <none> and password file <none>
I20240502 13:59:49.473479 29908 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:49.473536 29908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:49.473702 29908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:49.475344 29908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "06e5c7647a774d648861526ff1433622"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.475795 29908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "06e5c7647a774d648861526ff1433622"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.479424 29908 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:49.481635 29922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.482560 29908 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:49.482656 29908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "06e5c7647a774d648861526ff1433622"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.482746 29908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:49.507380 29908 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:49.508129 29908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:49.508296 29908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:49.508859 29908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:49.510068 29908 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:49.510120 29908 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.510159 29908 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:49.510180 29908 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.518404 29908 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.134:35775
I20240502 13:59:49.518432 30035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.134:35775 every 8 connection(s)
I20240502 13:59:49.519362 29908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 13:59:49.527957 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 29908
I20240502 13:59:49.528082 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 13:59:49.530231 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.135:0
--local_ip_for_outbound_sockets=127.26.25.135
--webserver_interface=127.26.25.135
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:49.533692 30036 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:49.533919 30036 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:49.534536 30036 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:49.535791 29187 ts_manager.cc:194] Registered new tserver with Master: 06e5c7647a774d648861526ff1433622 (127.26.25.134:35775)
I20240502 13:59:49.536702 29187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.134:47161
I20240502 13:59:49.616304 29378 heartbeater.cc:503] Master 127.26.25.190:33009 was elected leader, sending a full tablet report...
W20240502 13:59:49.639024 30039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:49.639257 30039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:49.639334 30039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:49.642441 30039 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:49.642517 30039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:49.642616 30039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.135
I20240502 13:59:49.646229 30039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.26.25.135
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.30039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.135
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:49.647212 30039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:49.648378 30039 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:49.656647 30045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.657110 30047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.657735 30044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:49.658627 30039 server_base.cc:1034] running on GCE node
I20240502 13:59:49.658989 30039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:49.659529 30039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:49.660678 30039 hybrid_clock.cc:648] HybridClock initialized: now 1714658389660655 us; error 39 us; skew 500 ppm
I20240502 13:59:49.662556 30039 webserver.cc:460] Webserver started at http://127.26.25.135:38449/ using document root <none> and password file <none>
I20240502 13:59:49.663086 30039 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:49.663143 30039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:49.663323 30039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:49.665117 30039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "374bd621a26249a2b9e4744235a88c2d"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.665588 30039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "374bd621a26249a2b9e4744235a88c2d"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.669642 30039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20240502 13:59:49.671973 30053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.673521 30039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:49.673626 30039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "374bd621a26249a2b9e4744235a88c2d"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:49.673712 30039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:49.691435 30039 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:49.692157 30039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:49.692315 30039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:49.692854 30039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:49.693737 30039 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:49.693786 30039 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.693846 30039 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:49.693872 30039 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.702726 30039 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.135:40775
I20240502 13:59:49.702762 30166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.135:40775 every 8 connection(s)
I20240502 13:59:49.703625 30039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 13:59:49.706241 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 30039
I20240502 13:59:49.706355 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 13:59:49.712632 30167 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:49.712841 30167 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:49.713327 30167 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:49.714229 29187 ts_manager.cc:194] Registered new tserver with Master: 374bd621a26249a2b9e4744235a88c2d (127.26.25.135:40775)
I20240502 13:59:49.714774 29187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.135:35595
I20240502 13:59:49.720984 26726 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 13:59:49.729743 29187 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51064:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 13:59:49.746093 29575 tablet_service.cc:1467] Processing CreateTablet for tablet 418124b2a16a49a8aaf0e50473b9bf50 (DEFAULT_TABLE table=TestTable [id=7fb6c077c1894953ada9e0c980e995eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:49.747022 29575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 418124b2a16a49a8aaf0e50473b9bf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:49.752125 30181 tablet_bootstrap.cc:492] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: Bootstrap starting.
I20240502 13:59:49.754155 30181 tablet_bootstrap.cc:654] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:49.754838 30181 log.cc:824] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:49.756636 30181 tablet_bootstrap.cc:492] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: No bootstrap required, opened a new log
I20240502 13:59:49.756824 30181 ts_tablet_manager.cc:1367] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 13:59:49.759222 30181 raft_consensus.cc:348] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } }
I20240502 13:59:49.759394 30181 raft_consensus.cc:374] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:49.759449 30181 raft_consensus.cc:729] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5265d55315f5413bb45818121d568e8b, State: Initialized, Role: FOLLOWER
I20240502 13:59:49.759853 30181 consensus_queue.cc:260] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } }
I20240502 13:59:49.759992 30181 raft_consensus.cc:388] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:49.760051 30181 raft_consensus.cc:482] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:49.760152 30181 raft_consensus.cc:3037] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:49.767299 30181 raft_consensus.cc:504] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } }
I20240502 13:59:49.767729 30181 leader_election.cc:302] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5265d55315f5413bb45818121d568e8b; no voters:
I20240502 13:59:49.768132 30181 leader_election.cc:288] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:49.768266 30183 raft_consensus.cc:2781] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:49.768507 30183 raft_consensus.cc:686] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 1 LEADER]: Becoming Leader. State: Replica: 5265d55315f5413bb45818121d568e8b, State: Running, Role: LEADER
I20240502 13:59:49.768887 29642 heartbeater.cc:503] Master 127.26.25.190:33009 was elected leader, sending a full tablet report...
I20240502 13:59:49.768838 30183 consensus_queue.cc:237] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } }
I20240502 13:59:49.769136 30181 ts_tablet_manager.cc:1398] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20240502 13:59:49.772816 29187 catalog_manager.cc:5484] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5265d55315f5413bb45818121d568e8b (127.26.25.131). New cstate: current_term: 1 leader_uuid: "5265d55315f5413bb45818121d568e8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:49.782187 26726 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 13:59:49.783550 26726 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 506b845ddf0a46ae9fc6a5ce4cafcc8a to finish bootstrapping
I20240502 13:59:49.791316 26726 ts_itest-base.cc:233] Waiting for 0 tablets on tserver a2d7d0121f2644188e2945b5ec6bf59e to finish bootstrapping
I20240502 13:59:49.796200 29510 heartbeater.cc:503] Master 127.26.25.190:33009 was elected leader, sending a full tablet report...
I20240502 13:59:49.796622 26726 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 5265d55315f5413bb45818121d568e8b to finish bootstrapping
I20240502 13:59:49.801031 26726 ts_itest-base.cc:233] Waiting for 0 tablets on tserver a18dcdc568884c3d9cf8dda212fc4034 to finish bootstrapping
I20240502 13:59:49.806131 26726 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 0832ab8d65cc47cabeb5e0128a14c151 to finish bootstrapping
I20240502 13:59:49.811178 26726 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 06e5c7647a774d648861526ff1433622 to finish bootstrapping
I20240502 13:59:49.816718 26726 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 374bd621a26249a2b9e4744235a88c2d to finish bootstrapping
I20240502 13:59:49.821580 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 29513
I20240502 13:59:49.827567 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 29645
I20240502 13:59:49.833087 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 29776
I20240502 13:59:49.838361 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 29908
I20240502 13:59:49.843892 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 30039
I20240502 13:59:53.600642 29187 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51064:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 13:59:53.612442 29443 tablet_service.cc:1467] Processing CreateTablet for tablet d4d2fe5bc85945659a881f83eff3fc23 (DEFAULT_TABLE table=rebalance_test_table_0 [id=616f782b4b1e4188a882d3553dce32a5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:53.612447 29311 tablet_service.cc:1467] Processing CreateTablet for tablet c714a72f483440ec9423d736b2a28786 (DEFAULT_TABLE table=rebalance_test_table_0 [id=616f782b4b1e4188a882d3553dce32a5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:53.612442 29312 tablet_service.cc:1467] Processing CreateTablet for tablet b3fe650164224a218b431f9a52c1283f (DEFAULT_TABLE table=rebalance_test_table_0 [id=616f782b4b1e4188a882d3553dce32a5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:53.613392 29311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c714a72f483440ec9423d736b2a28786. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:53.613411 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4d2fe5bc85945659a881f83eff3fc23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:53.613685 29312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3fe650164224a218b431f9a52c1283f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:53.618383 30203 tablet_bootstrap.cc:492] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Bootstrap starting.
I20240502 13:59:53.618551 30204 tablet_bootstrap.cc:492] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e: Bootstrap starting.
I20240502 13:59:53.620599 30203 tablet_bootstrap.cc:654] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:53.620599 30204 tablet_bootstrap.cc:654] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:53.621402 30204 log.cc:824] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:53.621402 30203 log.cc:824] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:53.622916 30203 tablet_bootstrap.cc:492] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: No bootstrap required, opened a new log
I20240502 13:59:53.623135 30203 ts_tablet_manager.cc:1367] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240502 13:59:53.623167 30204 tablet_bootstrap.cc:492] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e: No bootstrap required, opened a new log
I20240502 13:59:53.623386 30204 ts_tablet_manager.cc:1367] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 13:59:53.626286 30203 raft_consensus.cc:348] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.626461 30203 raft_consensus.cc:374] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:53.626389 30204 raft_consensus.cc:348] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.626535 30203 raft_consensus.cc:729] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Initialized, Role: FOLLOWER
I20240502 13:59:53.626547 30204 raft_consensus.cc:374] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:53.626603 30204 raft_consensus.cc:729] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Initialized, Role: FOLLOWER
I20240502 13:59:53.626940 30203 consensus_queue.cc:260] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.626969 30204 consensus_queue.cc:260] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.627063 30204 raft_consensus.cc:388] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:53.627067 30203 raft_consensus.cc:388] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:53.627132 30203 raft_consensus.cc:482] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:53.627130 30204 raft_consensus.cc:482] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:53.627230 30203 raft_consensus.cc:3037] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:53.627228 30204 raft_consensus.cc:3037] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:53.627980 30204 raft_consensus.cc:504] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.627996 30203 raft_consensus.cc:504] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.628263 30204 leader_election.cc:302] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 13:59:53.628263 30203 leader_election.cc:302] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:53.628531 30203 leader_election.cc:288] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:53.628544 30204 leader_election.cc:288] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:53.628614 30208 raft_consensus.cc:2781] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:53.628618 30207 raft_consensus.cc:2781] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:53.628787 30207 raft_consensus.cc:686] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Leader. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:53.628846 30208 raft_consensus.cc:686] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Leader. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 13:59:53.629127 30207 consensus_queue.cc:237] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.629140 30208 consensus_queue.cc:237] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.629331 30203 ts_tablet_manager.cc:1398] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240502 13:59:53.629330 30204 ts_tablet_manager.cc:1398] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20240502 13:59:53.629678 30203 tablet_bootstrap.cc:492] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Bootstrap starting.
I20240502 13:59:53.630616 30203 tablet_bootstrap.cc:654] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:53.631776 30203 tablet_bootstrap.cc:492] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: No bootstrap required, opened a new log
I20240502 13:59:53.631877 30203 ts_tablet_manager.cc:1367] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:53.631995 29187 catalog_manager.cc:5484] T d4d2fe5bc85945659a881f83eff3fc23 P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: term changed from 0 to 1, leader changed from <none> to a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130). New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:53.632261 30203 raft_consensus.cc:348] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.632319 30203 raft_consensus.cc:374] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:53.632339 30203 raft_consensus.cc:729] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Initialized, Role: FOLLOWER
I20240502 13:59:53.632432 30203 consensus_queue.cc:260] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.632536 30203 raft_consensus.cc:388] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:53.632570 30203 raft_consensus.cc:482] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:53.632607 30203 raft_consensus.cc:3037] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:53.633450 30203 raft_consensus.cc:504] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.633563 30203 leader_election.cc:302] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:53.633648 30203 leader_election.cc:288] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:53.633689 30210 raft_consensus.cc:2781] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:53.633841 30203 ts_tablet_manager.cc:1398] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 13:59:53.634891 29188 catalog_manager.cc:5484] T c714a72f483440ec9423d736b2a28786 P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129). New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:53.635041 30210 raft_consensus.cc:686] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Leader. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:53.635157 30210 consensus_queue.cc:237] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.638064 29187 catalog_manager.cc:5484] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129). New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } health_report { overall_health: HEALTHY } } }
W20240502 13:59:53.751207 30220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:53.751456 30220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:53.754426 30220 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
INSERT report
rows total: 2000
time total: 72.2023 ms
time per row: 0.0361012 ms
I20240502 13:59:53.867276 29182 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51064:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 13:59:53.872507 29443 tablet_service.cc:1467] Processing CreateTablet for tablet aa5712c33fbf4d8781478a12e76f584a (DEFAULT_TABLE table=rebalance_test_table_1 [id=f8339f00c8d2464c853e5f878b6a6a7c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:53.872553 29442 tablet_service.cc:1467] Processing CreateTablet for tablet ad3ff2489bad44a19600cc9dae475806 (DEFAULT_TABLE table=rebalance_test_table_1 [id=f8339f00c8d2464c853e5f878b6a6a7c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:53.872642 29312 tablet_service.cc:1467] Processing CreateTablet for tablet 576317299e6a4c15a742338c67292deb (DEFAULT_TABLE table=rebalance_test_table_1 [id=f8339f00c8d2464c853e5f878b6a6a7c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:53.872846 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5712c33fbf4d8781478a12e76f584a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:53.872890 29312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 576317299e6a4c15a742338c67292deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:53.872983 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad3ff2489bad44a19600cc9dae475806. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:53.874496 30204 tablet_bootstrap.cc:492] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: Bootstrap starting.
I20240502 13:59:53.874840 30203 tablet_bootstrap.cc:492] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Bootstrap starting.
I20240502 13:59:53.875387 30204 tablet_bootstrap.cc:654] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:53.875734 30203 tablet_bootstrap.cc:654] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:53.876492 30204 tablet_bootstrap.cc:492] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: No bootstrap required, opened a new log
I20240502 13:59:53.876564 30204 ts_tablet_manager.cc:1367] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:53.876802 30203 tablet_bootstrap.cc:492] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a: No bootstrap required, opened a new log
I20240502 13:59:53.876880 30203 ts_tablet_manager.cc:1367] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:53.877022 30204 raft_consensus.cc:348] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.877105 30204 raft_consensus.cc:374] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:53.877126 30204 raft_consensus.cc:729] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Initialized, Role: FOLLOWER
I20240502 13:59:53.877250 30204 consensus_queue.cc:260] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.877290 30203 raft_consensus.cc:348] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.877338 30204 raft_consensus.cc:388] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:53.877382 30204 raft_consensus.cc:482] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:53.877386 30203 raft_consensus.cc:374] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:53.877422 30204 raft_consensus.cc:3037] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:53.877418 30203 raft_consensus.cc:729] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Initialized, Role: FOLLOWER
I20240502 13:59:53.877532 30203 consensus_queue.cc:260] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.877600 30203 raft_consensus.cc:388] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:53.877626 30203 raft_consensus.cc:482] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:53.877666 30203 raft_consensus.cc:3037] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:53.878149 30204 raft_consensus.cc:504] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.878247 30204 leader_election.cc:302] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 13:59:53.878311 30204 leader_election.cc:288] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:53.878399 30208 raft_consensus.cc:2781] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:53.878372 30203 raft_consensus.cc:504] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.878500 30208 raft_consensus.cc:686] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Leader. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 13:59:53.878494 30203 leader_election.cc:302] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:53.878553 30204 ts_tablet_manager.cc:1398] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:53.878582 30203 leader_election.cc:288] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:53.878614 30208 consensus_queue.cc:237] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.878659 30218 raft_consensus.cc:2781] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:53.878697 30204 tablet_bootstrap.cc:492] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: Bootstrap starting.
I20240502 13:59:53.879091 30218 raft_consensus.cc:686] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Leader. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:53.879226 30218 consensus_queue.cc:237] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:53.879367 30203 ts_tablet_manager.cc:1398] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:53.879550 30204 tablet_bootstrap.cc:654] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:53.880507 29182 catalog_manager.cc:5484] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: term changed from 0 to 1, leader changed from <none> to a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130). New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:53.880829 30204 tablet_bootstrap.cc:492] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: No bootstrap required, opened a new log
I20240502 13:59:53.880915 30204 ts_tablet_manager.cc:1367] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:53.880560 29184 catalog_manager.cc:5484] T 576317299e6a4c15a742338c67292deb P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129). New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:53.881346 30204 raft_consensus.cc:348] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.881430 30204 raft_consensus.cc:374] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:53.881475 30204 raft_consensus.cc:729] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Initialized, Role: FOLLOWER
I20240502 13:59:53.881579 30204 consensus_queue.cc:260] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.881657 30204 raft_consensus.cc:388] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:53.881693 30204 raft_consensus.cc:482] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:53.881739 30204 raft_consensus.cc:3037] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:53.882423 30204 raft_consensus.cc:504] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.882530 30204 leader_election.cc:302] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 13:59:53.882634 30204 leader_election.cc:288] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:53.882779 30209 raft_consensus.cc:2781] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:53.882864 30209 raft_consensus.cc:686] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Leader. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 13:59:53.882890 30204 ts_tablet_manager.cc:1398] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:53.882987 30209 consensus_queue.cc:237] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:53.884583 29184 catalog_manager.cc:5484] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: term changed from 0 to 1, leader changed from <none> to a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130). New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } health_report { overall_health: HEALTHY } } }
W20240502 13:59:53.993824 30260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:53.994036 30260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:53.997126 30260 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
INSERT report
rows total: 2000
time total: 55.722 ms
time per row: 0.027861 ms
I20240502 13:59:54.090849 29182 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51064:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 13:59:54.096210 29312 tablet_service.cc:1467] Processing CreateTablet for tablet 14bc58294f6a48fd9e4ef5ab2a52afea (DEFAULT_TABLE table=rebalance_test_table_2 [id=eed1063799324a01a6b0b87659e9b508]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.096222 29443 tablet_service.cc:1467] Processing CreateTablet for tablet 2e17ab6801dd44928a9ff5a5b46d12ed (DEFAULT_TABLE table=rebalance_test_table_2 [id=eed1063799324a01a6b0b87659e9b508]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.096244 29311 tablet_service.cc:1467] Processing CreateTablet for tablet 0b4f51ba273c44f5805d3744caa4aa3d (DEFAULT_TABLE table=rebalance_test_table_2 [id=eed1063799324a01a6b0b87659e9b508]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.096549 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e17ab6801dd44928a9ff5a5b46d12ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.096565 29312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14bc58294f6a48fd9e4ef5ab2a52afea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.096755 29311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b4f51ba273c44f5805d3744caa4aa3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.098342 30203 tablet_bootstrap.cc:492] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Bootstrap starting.
I20240502 13:59:54.098690 30204 tablet_bootstrap.cc:492] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: Bootstrap starting.
I20240502 13:59:54.099397 30203 tablet_bootstrap.cc:654] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.099591 30204 tablet_bootstrap.cc:654] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.100526 30203 tablet_bootstrap.cc:492] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: No bootstrap required, opened a new log
I20240502 13:59:54.100602 30203 ts_tablet_manager.cc:1367] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 13:59:54.100724 30204 tablet_bootstrap.cc:492] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: No bootstrap required, opened a new log
I20240502 13:59:54.100807 30204 ts_tablet_manager.cc:1367] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 13:59:54.101001 30203 raft_consensus.cc:348] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.101089 30203 raft_consensus.cc:374] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.101110 30203 raft_consensus.cc:729] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.101245 30203 consensus_queue.cc:260] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.101226 30204 raft_consensus.cc:348] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.101312 30203 raft_consensus.cc:388] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:54.101318 30204 raft_consensus.cc:374] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.101344 30203 raft_consensus.cc:482] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:54.101366 30204 raft_consensus.cc:729] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.101385 30203 raft_consensus.cc:3037] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.101495 30204 consensus_queue.cc:260] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.101589 30204 raft_consensus.cc:388] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:54.101621 30204 raft_consensus.cc:482] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:54.101656 30204 raft_consensus.cc:3037] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.102239 30203 raft_consensus.cc:504] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.102375 30203 leader_election.cc:302] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:54.102447 30203 leader_election.cc:288] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:54.102425 30204 raft_consensus.cc:504] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.102509 30210 raft_consensus.cc:2781] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.102528 30204 leader_election.cc:302] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 13:59:54.102579 30210 raft_consensus.cc:686] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Leader. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:54.102685 30210 consensus_queue.cc:237] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.102744 30203 ts_tablet_manager.cc:1398] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 13:59:54.102877 30203 tablet_bootstrap.cc:492] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Bootstrap starting.
I20240502 13:59:54.102846 30208 raft_consensus.cc:2781] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.102952 30208 raft_consensus.cc:686] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Leader. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 13:59:54.103078 30208 consensus_queue.cc:237] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.103539 30204 leader_election.cc:288] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:54.103744 30203 tablet_bootstrap.cc:654] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.103777 30204 ts_tablet_manager.cc:1398] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20240502 13:59:54.104653 29182 catalog_manager.cc:5484] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: term changed from 0 to 1, leader changed from <none> to a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130). New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:54.104665 29185 catalog_manager.cc:5484] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129). New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:54.105404 30203 tablet_bootstrap.cc:492] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: No bootstrap required, opened a new log
I20240502 13:59:54.105489 30203 ts_tablet_manager.cc:1367] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 13:59:54.105917 30203 raft_consensus.cc:348] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.106022 30203 raft_consensus.cc:374] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.106057 30203 raft_consensus.cc:729] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.106179 30203 consensus_queue.cc:260] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.106279 30203 raft_consensus.cc:388] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:54.106328 30203 raft_consensus.cc:482] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:54.106380 30203 raft_consensus.cc:3037] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.107133 30203 raft_consensus.cc:504] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.107249 30203 leader_election.cc:302] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:54.107339 30203 leader_election.cc:288] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:54.107393 30210 raft_consensus.cc:2781] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.107483 30210 raft_consensus.cc:686] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Leader. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:54.107597 30210 consensus_queue.cc:237] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.107491 30203 ts_tablet_manager.cc:1398] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:54.108873 29185 catalog_manager.cc:5484] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129). New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } health_report { overall_health: HEALTHY } } }
W20240502 13:59:54.219954 30294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:54.220188 30294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:54.223208 30294 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
INSERT report
rows total: 2000
time total: 68.6913 ms
time per row: 0.0343457 ms
I20240502 13:59:54.333348 29186 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51064:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 13:59:54.338678 29443 tablet_service.cc:1467] Processing CreateTablet for tablet dd53513bf2744b66b3da0d7548a04b38 (DEFAULT_TABLE table=rebalance_test_table_3 [id=5bc4c8c9d29c410bbfea486485d41cff]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.338762 29442 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb (DEFAULT_TABLE table=rebalance_test_table_3 [id=5bc4c8c9d29c410bbfea486485d41cff]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.339030 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd53513bf2744b66b3da0d7548a04b38. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.339253 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.339421 29311 tablet_service.cc:1467] Processing CreateTablet for tablet 4edf51e537524169b98d62c50947571a (DEFAULT_TABLE table=rebalance_test_table_3 [id=5bc4c8c9d29c410bbfea486485d41cff]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.339708 29311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4edf51e537524169b98d62c50947571a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.340916 30204 tablet_bootstrap.cc:492] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: Bootstrap starting.
I20240502 13:59:54.341708 30203 tablet_bootstrap.cc:492] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Bootstrap starting.
I20240502 13:59:54.342121 30204 tablet_bootstrap.cc:654] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.342398 30203 tablet_bootstrap.cc:654] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.343245 30204 tablet_bootstrap.cc:492] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: No bootstrap required, opened a new log
I20240502 13:59:54.343335 30204 ts_tablet_manager.cc:1367] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:54.343436 30203 tablet_bootstrap.cc:492] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: No bootstrap required, opened a new log
I20240502 13:59:54.343523 30203 ts_tablet_manager.cc:1367] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:54.343751 30204 raft_consensus.cc:348] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.343856 30204 raft_consensus.cc:374] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.343897 30204 raft_consensus.cc:729] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.343940 30203 raft_consensus.cc:348] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.344058 30203 raft_consensus.cc:374] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.344033 30204 consensus_queue.cc:260] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.344092 30203 raft_consensus.cc:729] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.344110 30204 raft_consensus.cc:388] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:54.344147 30204 raft_consensus.cc:482] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:54.344195 30204 raft_consensus.cc:3037] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.344226 30203 consensus_queue.cc:260] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.344314 30203 raft_consensus.cc:388] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:54.344357 30203 raft_consensus.cc:482] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:54.344408 30203 raft_consensus.cc:3037] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.344868 30204 raft_consensus.cc:504] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.344974 30204 leader_election.cc:302] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 13:59:54.345049 30203 raft_consensus.cc:504] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.345122 30204 leader_election.cc:288] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:54.345180 30203 leader_election.cc:302] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:54.345191 30209 raft_consensus.cc:2781] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.345332 30209 raft_consensus.cc:686] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Leader. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 13:59:54.345377 30204 ts_tablet_manager.cc:1398] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 13:59:54.345450 30209 consensus_queue.cc:237] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.345533 30204 tablet_bootstrap.cc:492] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: Bootstrap starting.
I20240502 13:59:54.345541 30207 raft_consensus.cc:2781] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.345613 30207 raft_consensus.cc:686] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Leader. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:54.345710 30207 consensus_queue.cc:237] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.346909 30204 tablet_bootstrap.cc:654] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.346932 30203 leader_election.cc:288] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:54.347236 30203 ts_tablet_manager.cc:1398] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:54.347208 29186 catalog_manager.cc:5484] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129). New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:54.347407 29185 catalog_manager.cc:5484] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: term changed from 0 to 1, leader changed from <none> to a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130). New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:54.348248 30204 tablet_bootstrap.cc:492] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: No bootstrap required, opened a new log
I20240502 13:59:54.348331 30204 ts_tablet_manager.cc:1367] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 13:59:54.348783 30204 raft_consensus.cc:348] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.348887 30204 raft_consensus.cc:374] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.348934 30204 raft_consensus.cc:729] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.349059 30204 consensus_queue.cc:260] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.349152 30204 raft_consensus.cc:388] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:54.349195 30204 raft_consensus.cc:482] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:54.349242 30204 raft_consensus.cc:3037] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.350054 30204 raft_consensus.cc:504] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.350179 30204 leader_election.cc:302] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 13:59:54.350271 30204 leader_election.cc:288] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:54.350468 30204 ts_tablet_manager.cc:1398] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240502 13:59:54.350919 30209 raft_consensus.cc:2781] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.351006 30209 raft_consensus.cc:686] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Leader. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 13:59:54.351094 30209 consensus_queue.cc:237] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.353009 29186 catalog_manager.cc:5484] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: term changed from 0 to 1, leader changed from <none> to a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130). New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } health_report { overall_health: HEALTHY } } }
W20240502 13:59:54.462113 30328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:54.462345 30328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:54.465337 30328 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
INSERT report
rows total: 2000
time total: 64.2155 ms
time per row: 0.0321078 ms
I20240502 13:59:54.568223 29185 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51064:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 13:59:54.573251 29311 tablet_service.cc:1467] Processing CreateTablet for tablet 8464e256d453478caf17bab1048efd78 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4af4b17be3404e158177eaa235830e75]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.573408 29442 tablet_service.cc:1467] Processing CreateTablet for tablet 715eca20d6774ced9b9dba8d831bf6ac (DEFAULT_TABLE table=rebalance_test_table_4 [id=4af4b17be3404e158177eaa235830e75]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.573596 29311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8464e256d453478caf17bab1048efd78. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.573699 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 715eca20d6774ced9b9dba8d831bf6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.573635 29312 tablet_service.cc:1467] Processing CreateTablet for tablet dad1f80730fa4ca0a26b8ae9f3ab67bf (DEFAULT_TABLE table=rebalance_test_table_4 [id=4af4b17be3404e158177eaa235830e75]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.573868 29312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dad1f80730fa4ca0a26b8ae9f3ab67bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.575277 30203 tablet_bootstrap.cc:492] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Bootstrap starting.
I20240502 13:59:54.575767 30204 tablet_bootstrap.cc:492] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e: Bootstrap starting.
I20240502 13:59:54.576247 30203 tablet_bootstrap.cc:654] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.576503 30204 tablet_bootstrap.cc:654] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.577338 30203 tablet_bootstrap.cc:492] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: No bootstrap required, opened a new log
I20240502 13:59:54.577412 30203 ts_tablet_manager.cc:1367] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 13:59:54.577502 30204 tablet_bootstrap.cc:492] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e: No bootstrap required, opened a new log
I20240502 13:59:54.577579 30204 ts_tablet_manager.cc:1367] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:54.577898 30203 raft_consensus.cc:348] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.578006 30203 raft_consensus.cc:374] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.577977 30204 raft_consensus.cc:348] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.578048 30203 raft_consensus.cc:729] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.578076 30204 raft_consensus.cc:374] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.578109 30204 raft_consensus.cc:729] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.578176 30203 consensus_queue.cc:260] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.578227 30204 consensus_queue.cc:260] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.578279 30203 raft_consensus.cc:388] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:54.578315 30204 raft_consensus.cc:388] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:54.578320 30203 raft_consensus.cc:482] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:54.578356 30204 raft_consensus.cc:482] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:54.578363 30203 raft_consensus.cc:3037] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.578410 30204 raft_consensus.cc:3037] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.579005 30203 raft_consensus.cc:504] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.579006 30204 raft_consensus.cc:504] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.579113 30203 leader_election.cc:302] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:54.579123 30204 leader_election.cc:302] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 13:59:54.579210 30204 leader_election.cc:288] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:54.579267 30203 leader_election.cc:288] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:54.579317 30210 raft_consensus.cc:2781] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.579380 30323 raft_consensus.cc:2781] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.579397 30210 raft_consensus.cc:686] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Leader. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:54.579457 30323 raft_consensus.cc:686] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Leader. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 13:59:54.579509 30210 consensus_queue.cc:237] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.579552 30203 ts_tablet_manager.cc:1398] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20240502 13:59:54.579569 30323 consensus_queue.cc:237] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } }
I20240502 13:59:54.579747 30203 tablet_bootstrap.cc:492] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Bootstrap starting.
I20240502 13:59:54.580142 30204 ts_tablet_manager.cc:1398] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:54.580612 30203 tablet_bootstrap.cc:654] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.581166 29185 catalog_manager.cc:5484] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129). New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:54.581488 29184 catalog_manager.cc:5484] T 715eca20d6774ced9b9dba8d831bf6ac P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: term changed from 0 to 1, leader changed from <none> to a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130). New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:54.582688 30203 tablet_bootstrap.cc:492] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: No bootstrap required, opened a new log
I20240502 13:59:54.582765 30203 ts_tablet_manager.cc:1367] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 13:59:54.583346 30203 raft_consensus.cc:348] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.583442 30203 raft_consensus.cc:374] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.583468 30203 raft_consensus.cc:729] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.583559 30203 consensus_queue.cc:260] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.583622 30203 raft_consensus.cc:388] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:54.583644 30203 raft_consensus.cc:482] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:54.583671 30203 raft_consensus.cc:3037] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.584498 30203 raft_consensus.cc:504] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.584616 30203 leader_election.cc:302] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:54.584712 30203 leader_election.cc:288] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:54.584977 30203 ts_tablet_manager.cc:1398] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:54.585125 30218 raft_consensus.cc:2781] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.585207 30218 raft_consensus.cc:686] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Leader. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:54.585300 30218 consensus_queue.cc:237] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } }
I20240502 13:59:54.586483 29184 catalog_manager.cc:5484] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129). New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } health_report { overall_health: HEALTHY } } }
W20240502 13:59:54.692350 30360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:54.692550 30360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:54.695415 30360 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
INSERT report
rows total: 2000
time total: 68.2787 ms
time per row: 0.0341394 ms
I20240502 13:59:54.806874 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.131:37575
--local_ip_for_outbound_sockets=127.26.25.131
--tserver_master_addrs=127.26.25.190:33009
--webserver_port=45807
--webserver_interface=127.26.25.131
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 13:59:54.914690 30387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:54.914947 30387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:54.914996 30387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:54.918155 30387 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:54.918217 30387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:54.918306 30387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.131
I20240502 13:59:54.921870 30387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.131:37575
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.26.25.131
--webserver_port=45807
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.30387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.131
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:54.922777 30387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:54.924026 30387 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240502 13:59:54.932726 30387 server_base.cc:1034] running on GCE node
W20240502 13:59:54.933256 30392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:54.933822 30395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:54.934581 30393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:54.935636 30387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:54.936280 30387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:54.937419 30387 hybrid_clock.cc:648] HybridClock initialized: now 1714658394937405 us; error 30 us; skew 500 ppm
I20240502 13:59:54.939231 30387 webserver.cc:460] Webserver started at http://127.26.25.131:45807/ using document root <none> and password file <none>
I20240502 13:59:54.939810 30387 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:54.939934 30387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:54.943428 30387 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:54.945348 30401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:54.946377 30387 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:54.946508 30387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "5265d55315f5413bb45818121d568e8b"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-sldb"
I20240502 13:59:54.946979 30387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:54.958212 30387 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:54.958760 30387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:54.958890 30387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:54.959327 30387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:54.960369 30408 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240502 13:59:54.963194 30387 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240502 13:59:54.963244 30387 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20240502 13:59:54.963299 30387 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240502 13:59:54.964818 30387 ts_tablet_manager.cc:590] Registered 1 tablets
I20240502 13:59:54.964860 30387 ts_tablet_manager.cc:569] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:54.965101 30408 tablet_bootstrap.cc:492] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: Bootstrap starting.
I20240502 13:59:54.975168 30387 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.131:37575
I20240502 13:59:54.975253 30515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.131:37575 every 8 connection(s)
I20240502 13:59:54.976357 30387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 13:59:54.979492 30408 log.cc:824] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:54.981619 30408 tablet_bootstrap.cc:492] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:54.981958 30408 tablet_bootstrap.cc:492] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: Bootstrap complete.
I20240502 13:59:54.982685 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 30387
I20240502 13:59:54.983134 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.132:38647
--local_ip_for_outbound_sockets=127.26.25.132
--tserver_master_addrs=127.26.25.190:33009
--webserver_port=43153
--webserver_interface=127.26.25.132
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:54.983136 30408 ts_tablet_manager.cc:1367] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.006s
I20240502 13:59:54.985314 30516 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:54.985564 30516 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:54.986102 30516 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:54.986264 30408 raft_consensus.cc:348] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } }
I20240502 13:59:54.986591 30408 raft_consensus.cc:729] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5265d55315f5413bb45818121d568e8b, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.987020 30408 consensus_queue.cc:260] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } }
I20240502 13:59:54.987164 30408 raft_consensus.cc:388] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:54.987244 30408 raft_consensus.cc:482] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:54.987362 30408 raft_consensus.cc:3037] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:54.989161 29188 ts_manager.cc:194] Re-registered known tserver with Master: 5265d55315f5413bb45818121d568e8b (127.26.25.131:37575)
I20240502 13:59:54.990278 29188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.131:52101
I20240502 13:59:55.025749 30408 raft_consensus.cc:504] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } }
I20240502 13:59:55.026230 30408 leader_election.cc:302] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5265d55315f5413bb45818121d568e8b; no voters:
I20240502 13:59:55.026597 30408 leader_election.cc:288] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [CANDIDATE]: Term 2 election: Requested vote from peers
I20240502 13:59:55.026901 30521 raft_consensus.cc:2781] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 2 FOLLOWER]: Leader election won for term 2
I20240502 13:59:55.027169 30521 raft_consensus.cc:686] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [term 2 LEADER]: Becoming Leader. State: Replica: 5265d55315f5413bb45818121d568e8b, State: Running, Role: LEADER
I20240502 13:59:55.027483 30408 ts_tablet_manager.cc:1398] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b: Time spent starting tablet: real 0.044s user 0.011s sys 0.000s
I20240502 13:59:55.027524 30521 consensus_queue.cc:237] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } }
I20240502 13:59:55.028008 30516 heartbeater.cc:503] Master 127.26.25.190:33009 was elected leader, sending a full tablet report...
I20240502 13:59:55.030288 29188 catalog_manager.cc:5484] T 418124b2a16a49a8aaf0e50473b9bf50 P 5265d55315f5413bb45818121d568e8b reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "5265d55315f5413bb45818121d568e8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } health_report { overall_health: HEALTHY } } }
W20240502 13:59:55.095070 30520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:55.095288 30520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:55.095361 30520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:55.098387 30520 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:55.098460 30520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:55.098563 30520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.132
I20240502 13:59:55.102200 30520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.132:38647
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.26.25.132
--webserver_port=43153
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.30520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.132
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:55.103078 30520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:55.104261 30520 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:55.112560 30530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:55.113332 30531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:55.113899 30533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:55.114632 30520 server_base.cc:1034] running on GCE node
I20240502 13:59:55.114997 30520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:55.115535 30520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:55.116678 30520 hybrid_clock.cc:648] HybridClock initialized: now 1714658395116661 us; error 33 us; skew 500 ppm
I20240502 13:59:55.118429 30520 webserver.cc:460] Webserver started at http://127.26.25.132:43153/ using document root <none> and password file <none>
I20240502 13:59:55.118973 30520 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:55.119056 30520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:55.122288 30520 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:55.124190 30539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.125152 30520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:55.125265 30520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "a18dcdc568884c3d9cf8dda212fc4034"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:55.125674 30520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:55.157810 30520 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:55.158469 30520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:55.158643 30520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:55.159181 30520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:55.160130 30520 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:55.160182 30520 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.160243 30520 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:55.160269 30520 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.168573 30520 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.132:38647
I20240502 13:59:55.168610 30652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.132:38647 every 8 connection(s)
I20240502 13:59:55.169553 30520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 13:59:55.173121 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 30520
I20240502 13:59:55.173502 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.133:44559
--local_ip_for_outbound_sockets=127.26.25.133
--tserver_master_addrs=127.26.25.190:33009
--webserver_port=43445
--webserver_interface=127.26.25.133
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:55.178236 30653 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:55.178438 30653 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:55.178931 30653 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:55.179931 29188 ts_manager.cc:194] Re-registered known tserver with Master: a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132:38647)
I20240502 13:59:55.180472 29188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.132:47041
W20240502 13:59:55.285446 30656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:55.285707 30656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:55.285779 30656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:55.289115 30656 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:55.289197 30656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:55.289855 30656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.133
I20240502 13:59:55.293876 30656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.133:44559
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.25.133
--webserver_port=43445
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.30656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.133
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:55.294873 30656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:55.296085 30656 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:55.304486 30664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:55.305035 30662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:55.305831 30661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:55.306664 30656 server_base.cc:1034] running on GCE node
I20240502 13:59:55.307044 30656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:55.307580 30656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:55.308727 30656 hybrid_clock.cc:648] HybridClock initialized: now 1714658395308703 us; error 40 us; skew 500 ppm
I20240502 13:59:55.310564 30656 webserver.cc:460] Webserver started at http://127.26.25.133:43445/ using document root <none> and password file <none>
I20240502 13:59:55.311111 30656 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:55.311195 30656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:55.314592 30656 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:55.316623 30670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.317706 30656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:55.317814 30656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "0832ab8d65cc47cabeb5e0128a14c151"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:55.318249 30656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:55.331275 30656 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:55.331925 30656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:55.332083 30656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:55.332594 30656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:55.333448 30656 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:55.333504 30656 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.333563 30656 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:55.333590 30656 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.341971 30656 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.133:44559
I20240502 13:59:55.342027 30783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.133:44559 every 8 connection(s)
I20240502 13:59:55.342973 30656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 13:59:55.350067 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 30656
I20240502 13:59:55.350541 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.134:35775
--local_ip_for_outbound_sockets=127.26.25.134
--tserver_master_addrs=127.26.25.190:33009
--webserver_port=39549
--webserver_interface=127.26.25.134
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:55.351641 30784 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:55.351917 30784 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:55.352501 30784 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:55.353572 29188 ts_manager.cc:194] Re-registered known tserver with Master: 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133:44559)
I20240502 13:59:55.354416 29188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.133:35823
W20240502 13:59:55.460052 30787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:55.460276 30787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:55.460323 30787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:55.463465 30787 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:55.463543 30787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:55.463611 30787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.134
I20240502 13:59:55.467268 30787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.134:35775
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.26.25.134
--webserver_port=39549
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.30787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.134
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:55.468204 30787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:55.469336 30787 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:55.477806 30793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:55.478309 30795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:55.478955 30792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:55.479912 30787 server_base.cc:1034] running on GCE node
I20240502 13:59:55.480332 30787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:55.480886 30787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:55.482033 30787 hybrid_clock.cc:648] HybridClock initialized: now 1714658395482023 us; error 27 us; skew 500 ppm
I20240502 13:59:55.484015 30787 webserver.cc:460] Webserver started at http://127.26.25.134:39549/ using document root <none> and password file <none>
I20240502 13:59:55.484601 30787 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:55.484678 30787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:55.488184 30787 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20240502 13:59:55.490150 30801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.491182 30787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:55.491297 30787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "06e5c7647a774d648861526ff1433622"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:55.491715 30787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:55.508101 30787 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:55.508745 30787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:55.508900 30787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:55.509411 30787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:55.510344 30787 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:55.510412 30787 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.510473 30787 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:55.510517 30787 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.521152 30787 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.134:35775
I20240502 13:59:55.521179 30914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.134:35775 every 8 connection(s)
I20240502 13:59:55.522197 30787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 13:59:55.528019 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 30787
I20240502 13:59:55.528468 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.135:40775
--local_ip_for_outbound_sockets=127.26.25.135
--tserver_master_addrs=127.26.25.190:33009
--webserver_port=38449
--webserver_interface=127.26.25.135
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:55.531628 30915 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:55.531872 30915 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:55.532438 30915 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:55.533445 29188 ts_manager.cc:194] Re-registered known tserver with Master: 06e5c7647a774d648861526ff1433622 (127.26.25.134:35775)
I20240502 13:59:55.534003 29188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.134:51187
W20240502 13:59:55.638968 30918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:55.639175 30918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:55.639225 30918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:55.642279 30918 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:55.642342 30918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:55.642424 30918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.135
I20240502 13:59:55.645931 30918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.135:40775
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.26.25.135
--webserver_port=38449
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.30918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.135
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:55.646801 30918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:55.648028 30918 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:55.656687 30923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:55.657176 30926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:55.657996 30924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:55.658774 30918 server_base.cc:1034] running on GCE node
I20240502 13:59:55.659154 30918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:55.659714 30918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:55.660874 30918 hybrid_clock.cc:648] HybridClock initialized: now 1714658395660856 us; error 37 us; skew 500 ppm
I20240502 13:59:55.662583 30918 webserver.cc:460] Webserver started at http://127.26.25.135:38449/ using document root <none> and password file <none>
I20240502 13:59:55.663121 30918 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:55.663199 30918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:55.666585 30918 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:55.668495 30932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.669492 30918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:55.669610 30918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "374bd621a26249a2b9e4744235a88c2d"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-sldb"
I20240502 13:59:55.670068 30918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:55.688526 30918 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:55.689167 30918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:55.689322 30918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:55.689824 30918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:55.690877 30918 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:55.690940 30918 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.691005 30918 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:55.691033 30918 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:55.699389 30918 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.135:40775
I20240502 13:59:55.699414 31045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.135:40775 every 8 connection(s)
I20240502 13:59:55.700526 30918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 13:59:55.704751 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 30918
I20240502 13:59:55.709014 31046 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 13:59:55.709260 31046 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:55.709940 31046 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 13:59:55.711002 29188 ts_manager.cc:194] Re-registered known tserver with Master: 374bd621a26249a2b9e4744235a88c2d (127.26.25.135:40775)
I20240502 13:59:55.711550 29188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.135:55369
I20240502 13:59:55.898576 30718 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:55.899554 29312 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 13:59:55.907297 30980 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:55.910415 30587 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:55.911114 30849 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:55.916847 29442 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:55.917551 30444 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:56.033600 30718 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:56.034096 30980 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:56.035496 30849 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:56.036338 29312 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 13:59:56.037016 30587 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:56.055172 30444 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:56.058921 29442 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:56.089958 29463 consensus_queue.cc:237] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: true } }
I20240502 13:59:56.091409 31136 raft_consensus.cc:2932] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: true } } }
I20240502 13:59:56.094336 29188 catalog_manager.cc:5484] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: config changed from index -1 to 4, NON_VOTER a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:56.096047 29399 consensus_peers.cc:485] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e -> Peer a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132:38647): Couldn't send request to peer a18dcdc568884c3d9cf8dda212fc4034. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ac7ce8a2d944c86b2aa54c6f746d1eb. This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:56.101219 29332 consensus_queue.cc:237] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: true } }
I20240502 13:59:56.102497 31150 raft_consensus.cc:2932] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: true } } }
I20240502 13:59:56.105216 29188 catalog_manager.cc:5484] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: config changed from index -1 to 4, NON_VOTER 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133) added. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:56.107457 29264 consensus_peers.cc:485] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a -> Peer 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133:44559): Couldn't send request to peer 0832ab8d65cc47cabeb5e0128a14c151. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b3fe650164224a218b431f9a52c1283f. This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:56.111868 29463 consensus_queue.cc:237] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: true } }
I20240502 13:59:56.112733 31137 raft_consensus.cc:2932] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 374bd621a26249a2b9e4744235a88c2d (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: true } } }
I20240502 13:59:56.115377 29188 catalog_manager.cc:5484] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: config changed from index -1 to 4, NON_VOTER 374bd621a26249a2b9e4744235a88c2d (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:56.116868 29396 consensus_peers.cc:485] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e -> Peer 374bd621a26249a2b9e4744235a88c2d (127.26.25.135:40775): Couldn't send request to peer 374bd621a26249a2b9e4744235a88c2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e17ab6801dd44928a9ff5a5b46d12ed. This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:56.122455 29332 consensus_queue.cc:237] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: true } }
I20240502 13:59:56.123365 31150 raft_consensus.cc:2932] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06e5c7647a774d648861526ff1433622 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: true } } }
I20240502 13:59:56.125767 29188 catalog_manager.cc:5484] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: config changed from index -1 to 4, NON_VOTER 06e5c7647a774d648861526ff1433622 (127.26.25.134) added. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:56.127948 29267 consensus_peers.cc:485] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a -> Peer 06e5c7647a774d648861526ff1433622 (127.26.25.134:35775): Couldn't send request to peer 06e5c7647a774d648861526ff1433622. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 14bc58294f6a48fd9e4ef5ab2a52afea. This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:56.131953 29463 consensus_queue.cc:237] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: true } }
I20240502 13:59:56.132772 31136 raft_consensus.cc:2932] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 5265d55315f5413bb45818121d568e8b (127.26.25.131) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: true } } }
I20240502 13:59:56.134981 29188 catalog_manager.cc:5484] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: config changed from index -1 to 4, NON_VOTER 5265d55315f5413bb45818121d568e8b (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:56.136801 29398 consensus_peers.cc:485] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e -> Peer 5265d55315f5413bb45818121d568e8b (127.26.25.131:37575): Couldn't send request to peer 5265d55315f5413bb45818121d568e8b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aa5712c33fbf4d8781478a12e76f584a. This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:56.142037 29332 consensus_queue.cc:237] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: true } }
I20240502 13:59:56.142894 31149 raft_consensus.cc:2932] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: true } } }
I20240502 13:59:56.145013 29188 catalog_manager.cc:5484] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: config changed from index -1 to 4, NON_VOTER a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:56.147879 29267 consensus_peers.cc:485] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a -> Peer a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132:38647): Couldn't send request to peer a18dcdc568884c3d9cf8dda212fc4034. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8464e256d453478caf17bab1048efd78. This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:56.151132 29463 consensus_queue.cc:237] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: true } }
I20240502 13:59:56.152024 31137 raft_consensus.cc:2932] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: true } } }
I20240502 13:59:56.154402 29188 catalog_manager.cc:5484] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: config changed from index -1 to 4, NON_VOTER 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133) added. New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:56.155704 29396 consensus_peers.cc:485] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e -> Peer 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133:44559): Couldn't send request to peer 0832ab8d65cc47cabeb5e0128a14c151. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ad3ff2489bad44a19600cc9dae475806. This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:56.160945 29332 consensus_queue.cc:237] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: true } }
I20240502 13:59:56.161758 31150 raft_consensus.cc:2932] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 374bd621a26249a2b9e4744235a88c2d (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: true } } }
I20240502 13:59:56.164333 29188 catalog_manager.cc:5484] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: config changed from index -1 to 4, NON_VOTER 374bd621a26249a2b9e4744235a88c2d (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:56.165570 29264 consensus_peers.cc:485] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a -> Peer 374bd621a26249a2b9e4744235a88c2d (127.26.25.135:40775): Couldn't send request to peer 374bd621a26249a2b9e4744235a88c2d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dad1f80730fa4ca0a26b8ae9f3ab67bf. This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:56.170418 29332 consensus_queue.cc:237] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: true } }
I20240502 13:59:56.171201 31149 raft_consensus.cc:2932] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06e5c7647a774d648861526ff1433622 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: true } } }
I20240502 13:59:56.173434 29188 catalog_manager.cc:5484] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: config changed from index -1 to 4, NON_VOTER 06e5c7647a774d648861526ff1433622 (127.26.25.134) added. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:56.176396 29267 consensus_peers.cc:485] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a -> Peer 06e5c7647a774d648861526ff1433622 (127.26.25.134:35775): Couldn't send request to peer 06e5c7647a774d648861526ff1433622. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4edf51e537524169b98d62c50947571a. This is attempt 1: this message will repeat every 5th retry.
I20240502 13:59:56.182248 30653 heartbeater.cc:503] Master 127.26.25.190:33009 was elected leader, sending a full tablet report...
I20240502 13:59:56.356389 30784 heartbeater.cc:503] Master 127.26.25.190:33009 was elected leader, sending a full tablet report...
I20240502 13:59:56.527850 31284 ts_tablet_manager.cc:902] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034: Initiating tablet copy from peer a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699)
I20240502 13:59:56.528599 31284 tablet_copy_client.cc:308] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35699
I20240502 13:59:56.532368 29484 tablet_copy_service.cc:140] P a2d7d0121f2644188e2945b5ec6bf59e: Received BeginTabletCopySession request for tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb from peer a18dcdc568884c3d9cf8dda212fc4034 ({username='slave'} at 127.26.25.132:52573)
I20240502 13:59:56.532541 29484 tablet_copy_service.cc:161] P a2d7d0121f2644188e2945b5ec6bf59e: Beginning new tablet copy session on tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb from peer a18dcdc568884c3d9cf8dda212fc4034 at {username='slave'} at 127.26.25.132:52573: session id = a18dcdc568884c3d9cf8dda212fc4034-4ac7ce8a2d944c86b2aa54c6f746d1eb
I20240502 13:59:56.533660 29484 tablet_copy_source_session.cc:205] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 13:59:56.536551 30915 heartbeater.cc:503] Master 127.26.25.190:33009 was elected leader, sending a full tablet report...
I20240502 13:59:56.536938 31284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.542104 31284 tablet_copy_client.cc:783] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034: tablet copy: Starting download of 0 data blocks...
I20240502 13:59:56.542290 31284 tablet_copy_client.cc:647] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034: tablet copy: Starting download of 1 WAL segments...
I20240502 13:59:56.544226 31284 tablet_copy_client.cc:523] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 13:59:56.545779 31284 tablet_bootstrap.cc:492] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034: Bootstrap starting.
I20240502 13:59:56.551380 31284 log.cc:824] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:56.553488 31291 ts_tablet_manager.cc:902] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151: Initiating tablet copy from peer 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 13:59:56.554200 31291 tablet_copy_client.cc:308] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:35111
I20240502 13:59:56.558085 29352 tablet_copy_service.cc:140] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Received BeginTabletCopySession request for tablet b3fe650164224a218b431f9a52c1283f from peer 0832ab8d65cc47cabeb5e0128a14c151 ({username='slave'} at 127.26.25.133:48955)
I20240502 13:59:56.558238 29352 tablet_copy_service.cc:161] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Beginning new tablet copy session on tablet b3fe650164224a218b431f9a52c1283f from peer 0832ab8d65cc47cabeb5e0128a14c151 at {username='slave'} at 127.26.25.133:48955: session id = 0832ab8d65cc47cabeb5e0128a14c151-b3fe650164224a218b431f9a52c1283f
I20240502 13:59:56.559556 29352 tablet_copy_source_session.cc:205] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 13:59:56.562641 31291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3fe650164224a218b431f9a52c1283f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.567514 31291 tablet_copy_client.cc:783] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151: tablet copy: Starting download of 0 data blocks...
I20240502 13:59:56.567684 31291 tablet_copy_client.cc:647] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151: tablet copy: Starting download of 1 WAL segments...
I20240502 13:59:56.569447 31291 tablet_copy_client.cc:523] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 13:59:56.571158 31291 tablet_bootstrap.cc:492] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151: Bootstrap starting.
I20240502 13:59:56.572690 31284 tablet_bootstrap.cc:492] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:56.573037 31284 tablet_bootstrap.cc:492] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034: Bootstrap complete.
I20240502 13:59:56.573320 31284 ts_tablet_manager.cc:1367] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034: Time spent bootstrapping tablet: real 0.028s user 0.027s sys 0.000s
I20240502 13:59:56.575747 31284 raft_consensus.cc:348] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: true } }
I20240502 13:59:56.576001 31284 raft_consensus.cc:729] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a18dcdc568884c3d9cf8dda212fc4034, State: Initialized, Role: LEARNER
I20240502 13:59:56.576334 31284 consensus_queue.cc:260] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: true } }
I20240502 13:59:56.577214 31284 ts_tablet_manager.cc:1398] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 13:59:56.577311 31291 log.cc:824] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:56.578037 29484 tablet_copy_service.cc:342] P a2d7d0121f2644188e2945b5ec6bf59e: Request end of tablet copy session a18dcdc568884c3d9cf8dda212fc4034-4ac7ce8a2d944c86b2aa54c6f746d1eb received from {username='slave'} at 127.26.25.132:52573
I20240502 13:59:56.578146 29484 tablet_copy_service.cc:433] P a2d7d0121f2644188e2945b5ec6bf59e: ending tablet copy session a18dcdc568884c3d9cf8dda212fc4034-4ac7ce8a2d944c86b2aa54c6f746d1eb on tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb with peer a18dcdc568884c3d9cf8dda212fc4034
I20240502 13:59:56.595408 31291 tablet_bootstrap.cc:492] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:56.595738 31291 tablet_bootstrap.cc:492] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151: Bootstrap complete.
I20240502 13:59:56.596045 31291 ts_tablet_manager.cc:1367] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151: Time spent bootstrapping tablet: real 0.025s user 0.026s sys 0.000s
I20240502 13:59:56.597832 31291 raft_consensus.cc:348] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: true } }
I20240502 13:59:56.598021 31291 raft_consensus.cc:729] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0832ab8d65cc47cabeb5e0128a14c151, State: Initialized, Role: LEARNER
I20240502 13:59:56.598294 31291 consensus_queue.cc:260] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: true } }
I20240502 13:59:56.599058 31291 ts_tablet_manager.cc:1398] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 13:59:56.599844 29352 tablet_copy_service.cc:342] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Request end of tablet copy session 0832ab8d65cc47cabeb5e0128a14c151-b3fe650164224a218b431f9a52c1283f received from {username='slave'} at 127.26.25.133:48955
I20240502 13:59:56.599963 29352 tablet_copy_service.cc:433] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: ending tablet copy session 0832ab8d65cc47cabeb5e0128a14c151-b3fe650164224a218b431f9a52c1283f on tablet b3fe650164224a218b431f9a52c1283f with peer 0832ab8d65cc47cabeb5e0128a14c151
I20240502 13:59:56.618093 31291 ts_tablet_manager.cc:902] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151: Initiating tablet copy from peer a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699)
I20240502 13:59:56.618497 31291 tablet_copy_client.cc:308] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35699
I20240502 13:59:56.621568 29484 tablet_copy_service.cc:140] P a2d7d0121f2644188e2945b5ec6bf59e: Received BeginTabletCopySession request for tablet ad3ff2489bad44a19600cc9dae475806 from peer 0832ab8d65cc47cabeb5e0128a14c151 ({username='slave'} at 127.26.25.133:50117)
I20240502 13:59:56.621670 29484 tablet_copy_service.cc:161] P a2d7d0121f2644188e2945b5ec6bf59e: Beginning new tablet copy session on tablet ad3ff2489bad44a19600cc9dae475806 from peer 0832ab8d65cc47cabeb5e0128a14c151 at {username='slave'} at 127.26.25.133:50117: session id = 0832ab8d65cc47cabeb5e0128a14c151-ad3ff2489bad44a19600cc9dae475806
I20240502 13:59:56.622434 29484 tablet_copy_source_session.cc:205] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 13:59:56.623206 31291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad3ff2489bad44a19600cc9dae475806. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.624878 31291 tablet_copy_client.cc:783] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151: tablet copy: Starting download of 0 data blocks...
I20240502 13:59:56.625036 31291 tablet_copy_client.cc:647] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151: tablet copy: Starting download of 1 WAL segments...
I20240502 13:59:56.626060 31291 tablet_copy_client.cc:523] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 13:59:56.626933 31291 tablet_bootstrap.cc:492] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151: Bootstrap starting.
I20240502 13:59:56.642817 31291 tablet_bootstrap.cc:492] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:56.643096 31291 tablet_bootstrap.cc:492] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151: Bootstrap complete.
I20240502 13:59:56.643258 31291 ts_tablet_manager.cc:1367] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.002s
I20240502 13:59:56.643723 31291 raft_consensus.cc:348] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: true } }
I20240502 13:59:56.643857 31291 raft_consensus.cc:729] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0832ab8d65cc47cabeb5e0128a14c151, State: Initialized, Role: LEARNER
I20240502 13:59:56.643968 31291 consensus_queue.cc:260] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: true } }
I20240502 13:59:56.644241 31291 ts_tablet_manager.cc:1398] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151: Time spent starting tablet: real 0.001s user 0.000s sys 0.004s
I20240502 13:59:56.644877 29484 tablet_copy_service.cc:342] P a2d7d0121f2644188e2945b5ec6bf59e: Request end of tablet copy session 0832ab8d65cc47cabeb5e0128a14c151-ad3ff2489bad44a19600cc9dae475806 received from {username='slave'} at 127.26.25.133:50117
I20240502 13:59:56.644973 29484 tablet_copy_service.cc:433] P a2d7d0121f2644188e2945b5ec6bf59e: ending tablet copy session 0832ab8d65cc47cabeb5e0128a14c151-ad3ff2489bad44a19600cc9dae475806 on tablet ad3ff2489bad44a19600cc9dae475806 with peer 0832ab8d65cc47cabeb5e0128a14c151
I20240502 13:59:56.651684 31305 ts_tablet_manager.cc:902] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b: Initiating tablet copy from peer a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699)
I20240502 13:59:56.652369 31305 tablet_copy_client.cc:308] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35699
I20240502 13:59:56.654886 31311 ts_tablet_manager.cc:902] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622: Initiating tablet copy from peer 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 13:59:56.655602 31311 tablet_copy_client.cc:308] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:35111
I20240502 13:59:56.656857 29484 tablet_copy_service.cc:140] P a2d7d0121f2644188e2945b5ec6bf59e: Received BeginTabletCopySession request for tablet aa5712c33fbf4d8781478a12e76f584a from peer 5265d55315f5413bb45818121d568e8b ({username='slave'} at 127.26.25.131:45875)
I20240502 13:59:56.656993 29484 tablet_copy_service.cc:161] P a2d7d0121f2644188e2945b5ec6bf59e: Beginning new tablet copy session on tablet aa5712c33fbf4d8781478a12e76f584a from peer 5265d55315f5413bb45818121d568e8b at {username='slave'} at 127.26.25.131:45875: session id = 5265d55315f5413bb45818121d568e8b-aa5712c33fbf4d8781478a12e76f584a
I20240502 13:59:56.657826 29484 tablet_copy_source_session.cc:205] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 13:59:56.659524 31305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa5712c33fbf4d8781478a12e76f584a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.659739 29352 tablet_copy_service.cc:140] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Received BeginTabletCopySession request for tablet 14bc58294f6a48fd9e4ef5ab2a52afea from peer 06e5c7647a774d648861526ff1433622 ({username='slave'} at 127.26.25.134:42361)
I20240502 13:59:56.659862 29352 tablet_copy_service.cc:161] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Beginning new tablet copy session on tablet 14bc58294f6a48fd9e4ef5ab2a52afea from peer 06e5c7647a774d648861526ff1433622 at {username='slave'} at 127.26.25.134:42361: session id = 06e5c7647a774d648861526ff1433622-14bc58294f6a48fd9e4ef5ab2a52afea
I20240502 13:59:56.660522 29352 tablet_copy_source_session.cc:205] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 13:59:56.662290 31305 tablet_copy_client.cc:783] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b: tablet copy: Starting download of 0 data blocks...
I20240502 13:59:56.662480 31305 tablet_copy_client.cc:647] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b: tablet copy: Starting download of 1 WAL segments...
I20240502 13:59:56.663442 31311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14bc58294f6a48fd9e4ef5ab2a52afea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.663937 31305 tablet_copy_client.cc:523] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 13:59:56.665069 31305 tablet_bootstrap.cc:492] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b: Bootstrap starting.
I20240502 13:59:56.668813 31311 tablet_copy_client.cc:783] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622: tablet copy: Starting download of 0 data blocks...
I20240502 13:59:56.669009 31311 tablet_copy_client.cc:647] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622: tablet copy: Starting download of 1 WAL segments...
I20240502 13:59:56.670542 31311 tablet_copy_client.cc:523] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 13:59:56.672443 31311 tablet_bootstrap.cc:492] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622: Bootstrap starting.
I20240502 13:59:56.677915 31311 log.cc:824] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:56.690623 31334 ts_tablet_manager.cc:902] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d: Initiating tablet copy from peer 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 13:59:56.691475 31334 tablet_copy_client.cc:308] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:35111
I20240502 13:59:56.697095 31305 tablet_bootstrap.cc:492] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:56.697268 29352 tablet_copy_service.cc:140] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Received BeginTabletCopySession request for tablet dad1f80730fa4ca0a26b8ae9f3ab67bf from peer 374bd621a26249a2b9e4744235a88c2d ({username='slave'} at 127.26.25.135:52081)
I20240502 13:59:56.697365 29352 tablet_copy_service.cc:161] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Beginning new tablet copy session on tablet dad1f80730fa4ca0a26b8ae9f3ab67bf from peer 374bd621a26249a2b9e4744235a88c2d at {username='slave'} at 127.26.25.135:52081: session id = 374bd621a26249a2b9e4744235a88c2d-dad1f80730fa4ca0a26b8ae9f3ab67bf
I20240502 13:59:56.697443 31305 tablet_bootstrap.cc:492] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b: Bootstrap complete.
I20240502 13:59:56.697626 31305 ts_tablet_manager.cc:1367] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b: Time spent bootstrapping tablet: real 0.033s user 0.030s sys 0.000s
I20240502 13:59:56.698164 31305 raft_consensus.cc:348] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: true } }
I20240502 13:59:56.698263 29352 tablet_copy_source_session.cc:205] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 13:59:56.698280 31305 raft_consensus.cc:729] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5265d55315f5413bb45818121d568e8b, State: Initialized, Role: LEARNER
I20240502 13:59:56.698380 31305 consensus_queue.cc:260] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: true } }
I20240502 13:59:56.698757 31305 ts_tablet_manager.cc:1398] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 13:59:56.700901 29484 tablet_copy_service.cc:342] P a2d7d0121f2644188e2945b5ec6bf59e: Request end of tablet copy session 5265d55315f5413bb45818121d568e8b-aa5712c33fbf4d8781478a12e76f584a received from {username='slave'} at 127.26.25.131:45875
I20240502 13:59:56.700984 29484 tablet_copy_service.cc:433] P a2d7d0121f2644188e2945b5ec6bf59e: ending tablet copy session 5265d55315f5413bb45818121d568e8b-aa5712c33fbf4d8781478a12e76f584a on tablet aa5712c33fbf4d8781478a12e76f584a with peer 5265d55315f5413bb45818121d568e8b
I20240502 13:59:56.701910 31334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dad1f80730fa4ca0a26b8ae9f3ab67bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.702634 31346 ts_tablet_manager.cc:902] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d: Initiating tablet copy from peer a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699)
I20240502 13:59:56.703091 31346 tablet_copy_client.cc:308] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35699
I20240502 13:59:56.708701 29484 tablet_copy_service.cc:140] P a2d7d0121f2644188e2945b5ec6bf59e: Received BeginTabletCopySession request for tablet 2e17ab6801dd44928a9ff5a5b46d12ed from peer 374bd621a26249a2b9e4744235a88c2d ({username='slave'} at 127.26.25.135:50377)
I20240502 13:59:56.708730 31334 tablet_copy_client.cc:783] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d: tablet copy: Starting download of 0 data blocks...
I20240502 13:59:56.708820 29484 tablet_copy_service.cc:161] P a2d7d0121f2644188e2945b5ec6bf59e: Beginning new tablet copy session on tablet 2e17ab6801dd44928a9ff5a5b46d12ed from peer 374bd621a26249a2b9e4744235a88c2d at {username='slave'} at 127.26.25.135:50377: session id = 374bd621a26249a2b9e4744235a88c2d-2e17ab6801dd44928a9ff5a5b46d12ed
I20240502 13:59:56.708932 31334 tablet_copy_client.cc:647] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d: tablet copy: Starting download of 1 WAL segments...
I20240502 13:59:56.709631 29484 tablet_copy_source_session.cc:205] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 13:59:56.711680 31334 tablet_copy_client.cc:523] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 13:59:56.711838 31346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e17ab6801dd44928a9ff5a5b46d12ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.713452 31334 tablet_bootstrap.cc:492] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d: Bootstrap starting.
I20240502 13:59:56.713866 31046 heartbeater.cc:503] Master 127.26.25.190:33009 was elected leader, sending a full tablet report...
I20240502 13:59:56.714054 31346 tablet_copy_client.cc:783] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d: tablet copy: Starting download of 0 data blocks...
I20240502 13:59:56.714179 31346 tablet_copy_client.cc:647] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d: tablet copy: Starting download of 1 WAL segments...
I20240502 13:59:56.716094 31311 tablet_bootstrap.cc:492] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:56.716445 31311 tablet_bootstrap.cc:492] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622: Bootstrap complete.
I20240502 13:59:56.716742 31311 ts_tablet_manager.cc:1367] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622: Time spent bootstrapping tablet: real 0.044s user 0.030s sys 0.000s
I20240502 13:59:56.717087 31346 tablet_copy_client.cc:523] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 13:59:56.718159 31346 tablet_bootstrap.cc:492] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d: Bootstrap starting.
I20240502 13:59:56.719286 31311 raft_consensus.cc:348] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: true } }
I20240502 13:59:56.719516 31311 raft_consensus.cc:729] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06e5c7647a774d648861526ff1433622, State: Initialized, Role: LEARNER
I20240502 13:59:56.719662 31334 log.cc:824] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:56.719884 31311 consensus_queue.cc:260] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: true } }
I20240502 13:59:56.720747 31311 ts_tablet_manager.cc:1398] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 13:59:56.722575 29352 tablet_copy_service.cc:342] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Request end of tablet copy session 06e5c7647a774d648861526ff1433622-14bc58294f6a48fd9e4ef5ab2a52afea received from {username='slave'} at 127.26.25.134:42361
I20240502 13:59:56.722662 29352 tablet_copy_service.cc:433] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: ending tablet copy session 06e5c7647a774d648861526ff1433622-14bc58294f6a48fd9e4ef5ab2a52afea on tablet 14bc58294f6a48fd9e4ef5ab2a52afea with peer 06e5c7647a774d648861526ff1433622
I20240502 13:59:56.732540 31284 ts_tablet_manager.cc:902] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034: Initiating tablet copy from peer 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 13:59:56.733006 31284 tablet_copy_client.cc:308] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:35111
I20240502 13:59:56.737133 29352 tablet_copy_service.cc:140] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Received BeginTabletCopySession request for tablet 8464e256d453478caf17bab1048efd78 from peer a18dcdc568884c3d9cf8dda212fc4034 ({username='slave'} at 127.26.25.132:47573)
I20240502 13:59:56.737248 29352 tablet_copy_service.cc:161] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Beginning new tablet copy session on tablet 8464e256d453478caf17bab1048efd78 from peer a18dcdc568884c3d9cf8dda212fc4034 at {username='slave'} at 127.26.25.132:47573: session id = a18dcdc568884c3d9cf8dda212fc4034-8464e256d453478caf17bab1048efd78
I20240502 13:59:56.738124 29352 tablet_copy_source_session.cc:205] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 13:59:56.739037 31284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8464e256d453478caf17bab1048efd78. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.740804 31284 tablet_copy_client.cc:783] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034: tablet copy: Starting download of 0 data blocks...
I20240502 13:59:56.740970 31284 tablet_copy_client.cc:647] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034: tablet copy: Starting download of 1 WAL segments...
I20240502 13:59:56.742007 31284 tablet_copy_client.cc:523] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 13:59:56.742986 31284 tablet_bootstrap.cc:492] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034: Bootstrap starting.
I20240502 13:59:56.751134 31334 tablet_bootstrap.cc:492] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:56.751549 31334 tablet_bootstrap.cc:492] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d: Bootstrap complete.
I20240502 13:59:56.751906 31334 ts_tablet_manager.cc:1367] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d: Time spent bootstrapping tablet: real 0.039s user 0.030s sys 0.008s
I20240502 13:59:56.752152 31311 ts_tablet_manager.cc:902] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622: Initiating tablet copy from peer 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 13:59:56.752594 31311 tablet_copy_client.cc:308] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:35111
I20240502 13:59:56.753077 29352 tablet_copy_service.cc:140] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Received BeginTabletCopySession request for tablet 4edf51e537524169b98d62c50947571a from peer 06e5c7647a774d648861526ff1433622 ({username='slave'} at 127.26.25.134:42361)
I20240502 13:59:56.753197 29352 tablet_copy_service.cc:161] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Beginning new tablet copy session on tablet 4edf51e537524169b98d62c50947571a from peer 06e5c7647a774d648861526ff1433622 at {username='slave'} at 127.26.25.134:42361: session id = 06e5c7647a774d648861526ff1433622-4edf51e537524169b98d62c50947571a
I20240502 13:59:56.754011 29352 tablet_copy_source_session.cc:205] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 13:59:56.754021 31346 tablet_bootstrap.cc:492] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:56.754276 31346 tablet_bootstrap.cc:492] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d: Bootstrap complete.
I20240502 13:59:56.754421 31346 ts_tablet_manager.cc:1367] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d: Time spent bootstrapping tablet: real 0.036s user 0.026s sys 0.000s
I20240502 13:59:56.754765 31311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4edf51e537524169b98d62c50947571a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.754629 31334 raft_consensus.cc:348] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: true } }
I20240502 13:59:56.754874 31334 raft_consensus.cc:729] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 374bd621a26249a2b9e4744235a88c2d, State: Initialized, Role: LEARNER
I20240502 13:59:56.754918 31346 raft_consensus.cc:348] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: true } }
I20240502 13:59:56.754997 31346 raft_consensus.cc:729] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 374bd621a26249a2b9e4744235a88c2d, State: Initialized, Role: LEARNER
I20240502 13:59:56.755262 31346 consensus_queue.cc:260] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: true } }
I20240502 13:59:56.755262 31334 consensus_queue.cc:260] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: true } }
I20240502 13:59:56.756201 31334 ts_tablet_manager.cc:1398] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 13:59:56.756201 31346 ts_tablet_manager.cc:1398] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:56.756970 29484 tablet_copy_service.cc:342] P a2d7d0121f2644188e2945b5ec6bf59e: Request end of tablet copy session 374bd621a26249a2b9e4744235a88c2d-2e17ab6801dd44928a9ff5a5b46d12ed received from {username='slave'} at 127.26.25.135:50377
I20240502 13:59:56.756969 29352 tablet_copy_service.cc:342] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Request end of tablet copy session 374bd621a26249a2b9e4744235a88c2d-dad1f80730fa4ca0a26b8ae9f3ab67bf received from {username='slave'} at 127.26.25.135:52081
I20240502 13:59:56.757051 29352 tablet_copy_service.cc:433] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: ending tablet copy session 374bd621a26249a2b9e4744235a88c2d-dad1f80730fa4ca0a26b8ae9f3ab67bf on tablet dad1f80730fa4ca0a26b8ae9f3ab67bf with peer 374bd621a26249a2b9e4744235a88c2d
I20240502 13:59:56.757051 29484 tablet_copy_service.cc:433] P a2d7d0121f2644188e2945b5ec6bf59e: ending tablet copy session 374bd621a26249a2b9e4744235a88c2d-2e17ab6801dd44928a9ff5a5b46d12ed on tablet 2e17ab6801dd44928a9ff5a5b46d12ed with peer 374bd621a26249a2b9e4744235a88c2d
I20240502 13:59:56.757279 31311 tablet_copy_client.cc:783] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622: tablet copy: Starting download of 0 data blocks...
I20240502 13:59:56.757424 31311 tablet_copy_client.cc:647] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622: tablet copy: Starting download of 1 WAL segments...
I20240502 13:59:56.761059 31311 tablet_copy_client.cc:523] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 13:59:56.762264 31311 tablet_bootstrap.cc:492] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622: Bootstrap starting.
I20240502 13:59:56.768898 31284 tablet_bootstrap.cc:492] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:56.769181 31284 tablet_bootstrap.cc:492] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034: Bootstrap complete.
I20240502 13:59:56.769340 31284 ts_tablet_manager.cc:1367] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034: Time spent bootstrapping tablet: real 0.026s user 0.021s sys 0.000s
I20240502 13:59:56.769834 31284 raft_consensus.cc:348] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: true } }
I20240502 13:59:56.769937 31284 raft_consensus.cc:729] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a18dcdc568884c3d9cf8dda212fc4034, State: Initialized, Role: LEARNER
I20240502 13:59:56.770042 31284 consensus_queue.cc:260] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: true } }
I20240502 13:59:56.770308 31284 ts_tablet_manager.cc:1398] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 13:59:56.770779 29352 tablet_copy_service.cc:342] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Request end of tablet copy session a18dcdc568884c3d9cf8dda212fc4034-8464e256d453478caf17bab1048efd78 received from {username='slave'} at 127.26.25.132:47573
I20240502 13:59:56.770853 29352 tablet_copy_service.cc:433] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: ending tablet copy session a18dcdc568884c3d9cf8dda212fc4034-8464e256d453478caf17bab1048efd78 on tablet 8464e256d453478caf17bab1048efd78 with peer a18dcdc568884c3d9cf8dda212fc4034
I20240502 13:59:56.780495 31311 tablet_bootstrap.cc:492] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:56.780728 31311 tablet_bootstrap.cc:492] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622: Bootstrap complete.
I20240502 13:59:56.780859 31311 ts_tablet_manager.cc:1367] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622: Time spent bootstrapping tablet: real 0.019s user 0.018s sys 0.000s
I20240502 13:59:56.781313 31311 raft_consensus.cc:348] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: true } }
I20240502 13:59:56.781414 31311 raft_consensus.cc:729] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06e5c7647a774d648861526ff1433622, State: Initialized, Role: LEARNER
I20240502 13:59:56.781502 31311 consensus_queue.cc:260] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: true } }
I20240502 13:59:56.781740 31311 ts_tablet_manager.cc:1398] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:56.782289 29352 tablet_copy_service.cc:342] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Request end of tablet copy session 06e5c7647a774d648861526ff1433622-4edf51e537524169b98d62c50947571a received from {username='slave'} at 127.26.25.134:42361
I20240502 13:59:56.782369 29352 tablet_copy_service.cc:433] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: ending tablet copy session 06e5c7647a774d648861526ff1433622-4edf51e537524169b98d62c50947571a on tablet 4edf51e537524169b98d62c50947571a with peer 06e5c7647a774d648861526ff1433622
I20240502 13:59:56.944849 30607 raft_consensus.cc:1203] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 13:59:57.061681 30869 raft_consensus.cc:1203] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 13:59:57.090901 30738 raft_consensus.cc:1203] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 13:59:57.119844 30738 raft_consensus.cc:1203] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 13:59:57.158370 30461 raft_consensus.cc:1203] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 13:59:57.176499 30607 raft_consensus.cc:1203] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 13:59:57.188720 30869 raft_consensus.cc:1203] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 13:59:57.193403 31000 raft_consensus.cc:1203] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 13:59:57.298620 31000 raft_consensus.cc:1203] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 13:59:57.435240 31136 raft_consensus.cc:1050] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: attempting to promote NON_VOTER a18dcdc568884c3d9cf8dda212fc4034 to VOTER
I20240502 13:59:57.435791 31136 consensus_queue.cc:237] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } }
I20240502 13:59:57.436961 30607 raft_consensus.cc:1261] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [term 1 LEARNER]: Refusing update from remote peer a2d7d0121f2644188e2945b5ec6bf59e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 13:59:57.437371 31442 consensus_queue.cc:1035] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:57.439769 31136 raft_consensus.cc:2932] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } } }
I20240502 13:59:57.440295 30607 raft_consensus.cc:2932] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } } }
I20240502 13:59:57.442804 29188 catalog_manager.cc:5484] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 reported cstate change: config changed from index 4 to 5, a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } } }
I20240502 13:59:57.487272 29463 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4ac7ce8a2d944c86b2aa54c6f746d1eb"
dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43396
I20240502 13:59:57.487438 29463 raft_consensus.cc:595] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:57.612515 31442 raft_consensus.cc:1050] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: attempting to promote NON_VOTER 5265d55315f5413bb45818121d568e8b to VOTER
I20240502 13:59:57.612847 31442 consensus_queue.cc:237] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false } }
I20240502 13:59:57.613936 30461 raft_consensus.cc:1261] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [term 1 LEARNER]: Refusing update from remote peer a2d7d0121f2644188e2945b5ec6bf59e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 13:59:57.614369 31442 consensus_queue.cc:1035] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:57.616219 31394 raft_consensus.cc:2932] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 5265d55315f5413bb45818121d568e8b (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false } } }
I20240502 13:59:57.616667 30461 raft_consensus.cc:2932] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 5265d55315f5413bb45818121d568e8b (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false } } }
I20240502 13:59:57.618577 29179 catalog_manager.cc:5484] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: config changed from index 4 to 5, 5265d55315f5413bb45818121d568e8b (127.26.25.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:57.620124 31441 raft_consensus.cc:1050] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: attempting to promote NON_VOTER 0832ab8d65cc47cabeb5e0128a14c151 to VOTER
I20240502 13:59:57.620635 31441 consensus_queue.cc:237] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } }
I20240502 13:59:57.621743 30738 raft_consensus.cc:1261] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 LEARNER]: Refusing update from remote peer 506b845ddf0a46ae9fc6a5ce4cafcc8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 13:59:57.622236 31440 consensus_queue.cc:1035] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:57.624573 31441 raft_consensus.cc:2932] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } } }
I20240502 13:59:57.625097 30738 raft_consensus.cc:2932] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } } }
I20240502 13:59:57.626616 29188 catalog_manager.cc:5484] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: config changed from index 4 to 5, 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:57.657509 31150 raft_consensus.cc:1050] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: attempting to promote NON_VOTER 06e5c7647a774d648861526ff1433622 to VOTER
I20240502 13:59:57.657806 31150 consensus_queue.cc:237] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } }
I20240502 13:59:57.658875 30869 raft_consensus.cc:1261] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [term 1 LEARNER]: Refusing update from remote peer 506b845ddf0a46ae9fc6a5ce4cafcc8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 13:59:57.659194 31150 raft_consensus.cc:1050] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: attempting to promote NON_VOTER 374bd621a26249a2b9e4744235a88c2d to VOTER
I20240502 13:59:57.659289 31441 consensus_queue.cc:1035] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:57.659484 31150 consensus_queue.cc:237] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } }
I20240502 13:59:57.660552 31000 raft_consensus.cc:1261] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [term 1 LEARNER]: Refusing update from remote peer 506b845ddf0a46ae9fc6a5ce4cafcc8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 13:59:57.660920 31150 consensus_queue.cc:1035] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:57.661859 31441 raft_consensus.cc:2932] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06e5c7647a774d648861526ff1433622 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } } }
I20240502 13:59:57.662433 30869 raft_consensus.cc:2932] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06e5c7647a774d648861526ff1433622 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } } }
I20240502 13:59:57.663221 31441 raft_consensus.cc:2932] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 374bd621a26249a2b9e4744235a88c2d (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } } }
I20240502 13:59:57.664126 31000 raft_consensus.cc:2932] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 374bd621a26249a2b9e4744235a88c2d (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } } }
I20240502 13:59:57.664578 29179 catalog_manager.cc:5484] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: config changed from index 4 to 5, 06e5c7647a774d648861526ff1433622 (127.26.25.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:57.666496 29186 catalog_manager.cc:5484] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d reported cstate change: config changed from index 4 to 5, 374bd621a26249a2b9e4744235a88c2d (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } } }
I20240502 13:59:57.668743 31553 raft_consensus.cc:1050] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: attempting to promote NON_VOTER 06e5c7647a774d648861526ff1433622 to VOTER
I20240502 13:59:57.668984 31553 consensus_queue.cc:237] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } }
I20240502 13:59:57.669890 30869 raft_consensus.cc:1261] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [term 1 LEARNER]: Refusing update from remote peer 506b845ddf0a46ae9fc6a5ce4cafcc8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 13:59:57.670234 31553 consensus_queue.cc:1035] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:57.671737 31440 raft_consensus.cc:2932] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06e5c7647a774d648861526ff1433622 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } } }
I20240502 13:59:57.672163 30869 raft_consensus.cc:2932] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06e5c7647a774d648861526ff1433622 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } } }
I20240502 13:59:57.673832 29188 catalog_manager.cc:5484] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: config changed from index 4 to 5, 06e5c7647a774d648861526ff1433622 (127.26.25.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:57.682672 31394 raft_consensus.cc:1050] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: attempting to promote NON_VOTER 0832ab8d65cc47cabeb5e0128a14c151 to VOTER
I20240502 13:59:57.682950 31394 consensus_queue.cc:237] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } }
I20240502 13:59:57.683967 30738 raft_consensus.cc:1261] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 LEARNER]: Refusing update from remote peer a2d7d0121f2644188e2945b5ec6bf59e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 13:59:57.684305 31394 consensus_queue.cc:1035] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:57.685912 31136 raft_consensus.cc:2932] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } } }
I20240502 13:59:57.686372 30738 raft_consensus.cc:2932] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } } }
I20240502 13:59:57.688172 29186 catalog_manager.cc:5484] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: config changed from index 4 to 5, 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:57.708245 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dad1f80730fa4ca0a26b8ae9f3ab67bf"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55222
I20240502 13:59:57.708410 29332 raft_consensus.cc:595] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:57.717882 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b3fe650164224a218b431f9a52c1283f"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55244
I20240502 13:59:57.718014 29332 raft_consensus.cc:595] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:57.727135 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "14bc58294f6a48fd9e4ef5ab2a52afea"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55248
I20240502 13:59:57.727267 29332 raft_consensus.cc:595] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:57.728930 31440 raft_consensus.cc:1050] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: attempting to promote NON_VOTER a18dcdc568884c3d9cf8dda212fc4034 to VOTER
I20240502 13:59:57.729312 31440 consensus_queue.cc:237] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } }
I20240502 13:59:57.730522 30607 raft_consensus.cc:1261] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [term 1 LEARNER]: Refusing update from remote peer 506b845ddf0a46ae9fc6a5ce4cafcc8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 13:59:57.730912 31553 consensus_queue.cc:1035] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:57.732679 31440 raft_consensus.cc:2932] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } } }
I20240502 13:59:57.733376 30607 raft_consensus.cc:2932] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } } }
I20240502 13:59:57.735854 29188 catalog_manager.cc:5484] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: config changed from index 4 to 5, a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:57.738720 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4edf51e537524169b98d62c50947571a"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55274
I20240502 13:59:57.738823 29332 raft_consensus.cc:595] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:57.744997 31442 raft_consensus.cc:1050] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: attempting to promote NON_VOTER 374bd621a26249a2b9e4744235a88c2d to VOTER
I20240502 13:59:57.745366 31442 consensus_queue.cc:237] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } }
I20240502 13:59:57.746457 31000 raft_consensus.cc:1261] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [term 1 LEARNER]: Refusing update from remote peer a2d7d0121f2644188e2945b5ec6bf59e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 13:59:57.746837 31442 consensus_queue.cc:1035] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Connected to new peer: Peer: permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 13:59:57.748658 29463 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ad3ff2489bad44a19600cc9dae475806"
dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43450
I20240502 13:59:57.748780 29463 raft_consensus.cc:595] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:57.748698 31136 raft_consensus.cc:2932] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 374bd621a26249a2b9e4744235a88c2d (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } } }
I20240502 13:59:57.750587 31000 raft_consensus.cc:2932] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 374bd621a26249a2b9e4744235a88c2d (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } } }
I20240502 13:59:57.751050 29186 catalog_manager.cc:5484] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: config changed from index 4 to 5, 374bd621a26249a2b9e4744235a88c2d (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:57.759407 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8464e256d453478caf17bab1048efd78"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55290
I20240502 13:59:57.759528 29332 raft_consensus.cc:595] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:57.768632 29463 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4ac7ce8a2d944c86b2aa54c6f746d1eb"
dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43474
I20240502 13:59:57.768750 29463 raft_consensus.cc:595] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:57.778124 29463 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2e17ab6801dd44928a9ff5a5b46d12ed"
dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43492
I20240502 13:59:57.778244 29463 raft_consensus.cc:595] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:57.790824 29463 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aa5712c33fbf4d8781478a12e76f584a"
dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43514
I20240502 13:59:57.790982 29463 raft_consensus.cc:595] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:58.001852 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dad1f80730fa4ca0a26b8ae9f3ab67bf"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55316
I20240502 13:59:58.001989 29332 raft_consensus.cc:595] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:58.011744 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b3fe650164224a218b431f9a52c1283f"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55342
I20240502 13:59:58.011916 29332 raft_consensus.cc:595] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:58.021150 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "14bc58294f6a48fd9e4ef5ab2a52afea"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55366
I20240502 13:59:58.021266 29332 raft_consensus.cc:595] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:58.026932 31440 raft_consensus.cc:979] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: : Instructing follower 0832ab8d65cc47cabeb5e0128a14c151 to start an election
I20240502 13:59:58.027031 31440 raft_consensus.cc:1067] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Signalling peer 0832ab8d65cc47cabeb5e0128a14c151 to start an election
I20240502 13:59:58.027664 30738 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b3fe650164224a218b431f9a52c1283f"
dest_uuid: "0832ab8d65cc47cabeb5e0128a14c151"
from {username='slave'} at 127.26.25.129:52279
I20240502 13:59:58.027900 30738 raft_consensus.cc:482] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 13:59:58.028021 30738 raft_consensus.cc:3037] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.029134 30738 raft_consensus.cc:504] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } }
I20240502 13:59:58.029944 30738 leader_election.cc:288] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [CANDIDATE]: Term 2 election: Requested vote from peers 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 13:59:58.030333 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3fe650164224a218b431f9a52c1283f" candidate_uuid: "0832ab8d65cc47cabeb5e0128a14c151" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
I20240502 13:59:58.030485 29332 raft_consensus.cc:3032] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 13:59:58.030534 29332 raft_consensus.cc:729] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:58.030689 29332 consensus_queue.cc:260] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } }
I20240502 13:59:58.030845 29332 raft_consensus.cc:3037] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.031924 29332 raft_consensus.cc:2454] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0832ab8d65cc47cabeb5e0128a14c151 in term 2.
I20240502 13:59:58.032315 30673 leader_election.cc:302] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 0832ab8d65cc47cabeb5e0128a14c151, 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:58.032585 31548 raft_consensus.cc:2781] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 13:59:58.032757 31548 raft_consensus.cc:686] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [term 2 LEADER]: Becoming Leader. State: Replica: 0832ab8d65cc47cabeb5e0128a14c151, State: Running, Role: LEADER
I20240502 13:59:58.033056 31548 consensus_queue.cc:237] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } }
I20240502 13:59:58.033573 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4edf51e537524169b98d62c50947571a"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55394
I20240502 13:59:58.033668 29332 raft_consensus.cc:595] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:58.035151 29188 catalog_manager.cc:5484] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 reported cstate change: term changed from 1 to 2, leader changed from 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) to 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133). New cstate: current_term: 2 leader_uuid: "0832ab8d65cc47cabeb5e0128a14c151" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20240502 13:59:58.042604 29172 catalog_manager.cc:5014] ChangeConfig:REMOVE_PEER RPC for tablet b3fe650164224a218b431f9a52c1283f with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133:44559) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20240502 13:59:58.044593 29463 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ad3ff2489bad44a19600cc9dae475806"
dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43536
I20240502 13:59:58.044726 29463 raft_consensus.cc:595] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:58.054672 31394 raft_consensus.cc:979] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: : Instructing follower a18dcdc568884c3d9cf8dda212fc4034 to start an election
I20240502 13:59:58.054816 31136 raft_consensus.cc:1067] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Signalling peer a18dcdc568884c3d9cf8dda212fc4034 to start an election
I20240502 13:59:58.054829 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8464e256d453478caf17bab1048efd78"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55400
I20240502 13:59:58.054931 29332 raft_consensus.cc:595] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:58.055444 30607 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4ac7ce8a2d944c86b2aa54c6f746d1eb"
dest_uuid: "a18dcdc568884c3d9cf8dda212fc4034"
from {username='slave'} at 127.26.25.130:49233
I20240502 13:59:58.055636 30607 raft_consensus.cc:482] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 13:59:58.055742 30607 raft_consensus.cc:3037] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.056639 30607 raft_consensus.cc:504] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } }
I20240502 13:59:58.057494 30607 leader_election.cc:288] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [CANDIDATE]: Term 2 election: Requested vote from peers a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699)
I20240502 13:59:58.058094 29463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac7ce8a2d944c86b2aa54c6f746d1eb" candidate_uuid: "a18dcdc568884c3d9cf8dda212fc4034" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
I20240502 13:59:58.058225 29463 raft_consensus.cc:3032] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 13:59:58.058266 29463 raft_consensus.cc:729] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 13:59:58.058393 29463 consensus_queue.cc:260] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } }
I20240502 13:59:58.058517 29463 raft_consensus.cc:3037] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.059255 29463 raft_consensus.cc:2454] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a18dcdc568884c3d9cf8dda212fc4034 in term 2.
I20240502 13:59:58.059633 30543 leader_election.cc:302] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: a18dcdc568884c3d9cf8dda212fc4034, a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 13:59:58.059906 31297 raft_consensus.cc:2781] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 13:59:58.060092 31297 raft_consensus.cc:686] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [term 2 LEADER]: Becoming Leader. State: Replica: a18dcdc568884c3d9cf8dda212fc4034, State: Running, Role: LEADER
I20240502 13:59:58.060372 31297 consensus_queue.cc:237] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } }
I20240502 13:59:58.062328 29188 catalog_manager.cc:5484] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 reported cstate change: term changed from 1 to 2, leader changed from a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) to a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "a18dcdc568884c3d9cf8dda212fc4034" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.073546 29463 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2e17ab6801dd44928a9ff5a5b46d12ed"
dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43568
I20240502 13:59:58.073661 29463 raft_consensus.cc:595] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:58.082520 29463 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aa5712c33fbf4d8781478a12e76f584a"
dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43586
I20240502 13:59:58.082641 29463 raft_consensus.cc:595] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Received request to transfer leadership
I20240502 13:59:58.106762 31394 raft_consensus.cc:979] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: : Instructing follower 5265d55315f5413bb45818121d568e8b to start an election
I20240502 13:59:58.106866 31394 raft_consensus.cc:1067] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Signalling peer 5265d55315f5413bb45818121d568e8b to start an election
I20240502 13:59:58.107295 30461 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aa5712c33fbf4d8781478a12e76f584a"
dest_uuid: "5265d55315f5413bb45818121d568e8b"
from {username='slave'} at 127.26.25.130:52559
I20240502 13:59:58.107455 30461 raft_consensus.cc:482] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 13:59:58.107506 30461 raft_consensus.cc:3037] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.108309 30461 raft_consensus.cc:504] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false } }
I20240502 13:59:58.108925 30461 leader_election.cc:288] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [CANDIDATE]: Term 2 election: Requested vote from peers a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699)
I20240502 13:59:58.109280 29463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa5712c33fbf4d8781478a12e76f584a" candidate_uuid: "5265d55315f5413bb45818121d568e8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
I20240502 13:59:58.109393 29463 raft_consensus.cc:3032] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 13:59:58.109441 29463 raft_consensus.cc:729] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 13:59:58.109588 29463 consensus_queue.cc:260] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false } }
I20240502 13:59:58.109717 29463 raft_consensus.cc:3037] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.110430 29463 raft_consensus.cc:2454] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5265d55315f5413bb45818121d568e8b in term 2.
I20240502 13:59:58.110780 30405 leader_election.cc:302] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 5265d55315f5413bb45818121d568e8b, a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 13:59:58.110973 31347 raft_consensus.cc:2781] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [term 2 FOLLOWER]: Leader election won for term 2
I20240502 13:59:58.111141 31347 raft_consensus.cc:686] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [term 2 LEADER]: Becoming Leader. State: Replica: 5265d55315f5413bb45818121d568e8b, State: Running, Role: LEADER
I20240502 13:59:58.111274 31347 consensus_queue.cc:237] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false } }
I20240502 13:59:58.112898 29188 catalog_manager.cc:5484] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b reported cstate change: term changed from 1 to 2, leader changed from a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) to 5265d55315f5413bb45818121d568e8b (127.26.25.131). New cstate: current_term: 2 leader_uuid: "5265d55315f5413bb45818121d568e8b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.138924 31440 raft_consensus.cc:979] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: : Instructing follower 06e5c7647a774d648861526ff1433622 to start an election
I20240502 13:59:58.139045 31440 raft_consensus.cc:1067] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Signalling peer 06e5c7647a774d648861526ff1433622 to start an election
I20240502 13:59:58.139526 30869 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4edf51e537524169b98d62c50947571a"
dest_uuid: "06e5c7647a774d648861526ff1433622"
from {username='slave'} at 127.26.25.129:41661
I20240502 13:59:58.139703 30869 raft_consensus.cc:482] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 13:59:58.139814 30869 raft_consensus.cc:3037] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.140637 30869 raft_consensus.cc:504] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } }
I20240502 13:59:58.141451 30869 leader_election.cc:288] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [CANDIDATE]: Term 2 election: Requested vote from peers 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 13:59:58.141819 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4edf51e537524169b98d62c50947571a" candidate_uuid: "06e5c7647a774d648861526ff1433622" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
I20240502 13:59:58.141937 29332 raft_consensus.cc:3032] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 13:59:58.141988 29332 raft_consensus.cc:729] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:58.142159 29332 consensus_queue.cc:260] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } }
I20240502 13:59:58.142297 29332 raft_consensus.cc:3037] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.143028 29332 raft_consensus.cc:2454] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06e5c7647a774d648861526ff1433622 in term 2.
I20240502 13:59:58.143401 30804 leader_election.cc:302] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 06e5c7647a774d648861526ff1433622, 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:58.143649 31365 raft_consensus.cc:2781] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 13:59:58.143848 31365 raft_consensus.cc:686] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [term 2 LEADER]: Becoming Leader. State: Replica: 06e5c7647a774d648861526ff1433622, State: Running, Role: LEADER
I20240502 13:59:58.144116 31365 consensus_queue.cc:237] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } }
I20240502 13:59:58.145893 29188 catalog_manager.cc:5484] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 reported cstate change: term changed from 1 to 2, leader changed from 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) to 06e5c7647a774d648861526ff1433622 (127.26.25.134). New cstate: current_term: 2 leader_uuid: "06e5c7647a774d648861526ff1433622" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.171248 31136 raft_consensus.cc:979] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: : Instructing follower 0832ab8d65cc47cabeb5e0128a14c151 to start an election
I20240502 13:59:58.171361 31136 raft_consensus.cc:1067] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Signalling peer 0832ab8d65cc47cabeb5e0128a14c151 to start an election
I20240502 13:59:58.171793 30738 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ad3ff2489bad44a19600cc9dae475806"
dest_uuid: "0832ab8d65cc47cabeb5e0128a14c151"
from {username='slave'} at 127.26.25.130:43805
I20240502 13:59:58.171960 30738 raft_consensus.cc:482] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 13:59:58.172025 30738 raft_consensus.cc:3037] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.172819 30738 raft_consensus.cc:504] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } }
I20240502 13:59:58.173168 30738 leader_election.cc:288] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [CANDIDATE]: Term 2 election: Requested vote from peers a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699)
I20240502 13:59:58.173599 29463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad3ff2489bad44a19600cc9dae475806" candidate_uuid: "0832ab8d65cc47cabeb5e0128a14c151" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
I20240502 13:59:58.173732 29463 raft_consensus.cc:3032] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 13:59:58.173780 29463 raft_consensus.cc:729] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 13:59:58.173918 29463 consensus_queue.cc:260] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } }
I20240502 13:59:58.174044 29463 raft_consensus.cc:3037] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.175000 29463 raft_consensus.cc:2454] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0832ab8d65cc47cabeb5e0128a14c151 in term 2.
I20240502 13:59:58.175279 30674 leader_election.cc:302] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 0832ab8d65cc47cabeb5e0128a14c151, a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 13:59:58.175458 31548 raft_consensus.cc:2781] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 13:59:58.175566 31548 raft_consensus.cc:686] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [term 2 LEADER]: Becoming Leader. State: Replica: 0832ab8d65cc47cabeb5e0128a14c151, State: Running, Role: LEADER
I20240502 13:59:58.175698 31548 consensus_queue.cc:237] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } }
I20240502 13:59:58.177124 29188 catalog_manager.cc:5484] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 reported cstate change: term changed from 1 to 2, leader changed from a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) to 0832ab8d65cc47cabeb5e0128a14c151 (127.26.25.133). New cstate: current_term: 2 leader_uuid: "0832ab8d65cc47cabeb5e0128a14c151" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.192828 31553 raft_consensus.cc:979] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: : Instructing follower 374bd621a26249a2b9e4744235a88c2d to start an election
I20240502 13:59:58.192942 31553 raft_consensus.cc:1067] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Signalling peer 374bd621a26249a2b9e4744235a88c2d to start an election
I20240502 13:59:58.193384 31000 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dad1f80730fa4ca0a26b8ae9f3ab67bf"
dest_uuid: "374bd621a26249a2b9e4744235a88c2d"
from {username='slave'} at 127.26.25.129:37907
I20240502 13:59:58.193567 31000 raft_consensus.cc:482] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 13:59:58.193678 31000 raft_consensus.cc:3037] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.194474 31000 raft_consensus.cc:504] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } }
I20240502 13:59:58.195369 31000 leader_election.cc:288] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [CANDIDATE]: Term 2 election: Requested vote from peers 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 13:59:58.195745 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dad1f80730fa4ca0a26b8ae9f3ab67bf" candidate_uuid: "374bd621a26249a2b9e4744235a88c2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
I20240502 13:59:58.195896 29332 raft_consensus.cc:3032] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 13:59:58.195946 29332 raft_consensus.cc:729] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:58.196090 29332 consensus_queue.cc:260] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } }
I20240502 13:59:58.196211 29332 raft_consensus.cc:3037] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.196918 29332 raft_consensus.cc:2454] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 374bd621a26249a2b9e4744235a88c2d in term 2.
I20240502 13:59:58.197225 30935 leader_election.cc:302] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 374bd621a26249a2b9e4744235a88c2d, 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:58.197453 31392 raft_consensus.cc:2781] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 13:59:58.197623 31392 raft_consensus.cc:686] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [term 2 LEADER]: Becoming Leader. State: Replica: 374bd621a26249a2b9e4744235a88c2d, State: Running, Role: LEADER
I20240502 13:59:58.197870 31392 consensus_queue.cc:237] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } }
I20240502 13:59:58.199606 29188 catalog_manager.cc:5484] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d reported cstate change: term changed from 1 to 2, leader changed from 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) to 374bd621a26249a2b9e4744235a88c2d (127.26.25.135). New cstate: current_term: 2 leader_uuid: "374bd621a26249a2b9e4744235a88c2d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.237275 31440 raft_consensus.cc:979] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: : Instructing follower 06e5c7647a774d648861526ff1433622 to start an election
I20240502 13:59:58.237381 31440 raft_consensus.cc:1067] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Signalling peer 06e5c7647a774d648861526ff1433622 to start an election
I20240502 13:59:58.237802 30869 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14bc58294f6a48fd9e4ef5ab2a52afea"
dest_uuid: "06e5c7647a774d648861526ff1433622"
from {username='slave'} at 127.26.25.129:41661
I20240502 13:59:58.237967 30869 raft_consensus.cc:482] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 13:59:58.238036 30869 raft_consensus.cc:3037] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.238843 30869 raft_consensus.cc:504] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } }
I20240502 13:59:58.239135 30869 leader_election.cc:288] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [CANDIDATE]: Term 2 election: Requested vote from peers 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 13:59:58.239526 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14bc58294f6a48fd9e4ef5ab2a52afea" candidate_uuid: "06e5c7647a774d648861526ff1433622" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
I20240502 13:59:58.239636 29332 raft_consensus.cc:3032] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 13:59:58.239696 29332 raft_consensus.cc:729] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:58.239885 29332 consensus_queue.cc:260] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } }
I20240502 13:59:58.240021 29332 raft_consensus.cc:3037] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.240761 29332 raft_consensus.cc:2454] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06e5c7647a774d648861526ff1433622 in term 2.
I20240502 13:59:58.241019 30804 leader_election.cc:302] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 06e5c7647a774d648861526ff1433622, 506b845ddf0a46ae9fc6a5ce4cafcc8a; no voters:
I20240502 13:59:58.241199 31365 raft_consensus.cc:2781] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 13:59:58.241304 31365 raft_consensus.cc:686] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [term 2 LEADER]: Becoming Leader. State: Replica: 06e5c7647a774d648861526ff1433622, State: Running, Role: LEADER
I20240502 13:59:58.241448 31365 consensus_queue.cc:237] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } }
I20240502 13:59:58.243057 29188 catalog_manager.cc:5484] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 reported cstate change: term changed from 1 to 2, leader changed from 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) to 06e5c7647a774d648861526ff1433622 (127.26.25.134). New cstate: current_term: 2 leader_uuid: "06e5c7647a774d648861526ff1433622" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.248826 31553 raft_consensus.cc:979] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: : Instructing follower a18dcdc568884c3d9cf8dda212fc4034 to start an election
I20240502 13:59:58.248906 31553 raft_consensus.cc:1067] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Signalling peer a18dcdc568884c3d9cf8dda212fc4034 to start an election
I20240502 13:59:58.249341 30607 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8464e256d453478caf17bab1048efd78"
dest_uuid: "a18dcdc568884c3d9cf8dda212fc4034"
from {username='slave'} at 127.26.25.129:42941
I20240502 13:59:58.249462 30607 raft_consensus.cc:482] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 13:59:58.249523 30607 raft_consensus.cc:3037] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.250298 30607 raft_consensus.cc:504] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } }
I20240502 13:59:58.250671 30607 leader_election.cc:288] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [CANDIDATE]: Term 2 election: Requested vote from peers 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 13:59:58.251013 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8464e256d453478caf17bab1048efd78" candidate_uuid: "a18dcdc568884c3d9cf8dda212fc4034" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
I20240502 13:59:58.251116 29332 raft_consensus.cc:3032] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 13:59:58.251159 29332 raft_consensus.cc:729] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 13:59:58.251305 29332 consensus_queue.cc:260] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } }
I20240502 13:59:58.251416 29332 raft_consensus.cc:3037] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.252379 29332 raft_consensus.cc:2454] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a18dcdc568884c3d9cf8dda212fc4034 in term 2.
I20240502 13:59:58.252661 30542 leader_election.cc:302] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 506b845ddf0a46ae9fc6a5ce4cafcc8a, a18dcdc568884c3d9cf8dda212fc4034; no voters:
I20240502 13:59:58.252835 31297 raft_consensus.cc:2781] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 13:59:58.252921 31297 raft_consensus.cc:686] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [term 2 LEADER]: Becoming Leader. State: Replica: a18dcdc568884c3d9cf8dda212fc4034, State: Running, Role: LEADER
I20240502 13:59:58.253082 31297 consensus_queue.cc:237] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } }
I20240502 13:59:58.254379 29188 catalog_manager.cc:5484] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 reported cstate change: term changed from 1 to 2, leader changed from 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) to a18dcdc568884c3d9cf8dda212fc4034 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "a18dcdc568884c3d9cf8dda212fc4034" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.275228 31394 raft_consensus.cc:979] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: : Instructing follower 374bd621a26249a2b9e4744235a88c2d to start an election
I20240502 13:59:58.275339 31394 raft_consensus.cc:1067] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Signalling peer 374bd621a26249a2b9e4744235a88c2d to start an election
I20240502 13:59:58.275765 31000 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2e17ab6801dd44928a9ff5a5b46d12ed"
dest_uuid: "374bd621a26249a2b9e4744235a88c2d"
from {username='slave'} at 127.26.25.130:35791
I20240502 13:59:58.275943 31000 raft_consensus.cc:482] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 13:59:58.276024 31000 raft_consensus.cc:3037] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.277005 31000 raft_consensus.cc:504] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } }
I20240502 13:59:58.277348 31000 leader_election.cc:288] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [CANDIDATE]: Term 2 election: Requested vote from peers a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699)
I20240502 13:59:58.277752 29463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e17ab6801dd44928a9ff5a5b46d12ed" candidate_uuid: "374bd621a26249a2b9e4744235a88c2d" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
I20240502 13:59:58.277859 29463 raft_consensus.cc:3032] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 13:59:58.277904 29463 raft_consensus.cc:729] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 13:59:58.278048 29463 consensus_queue.cc:260] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } }
I20240502 13:59:58.278169 29463 raft_consensus.cc:3037] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Advancing to term 2
I20240502 13:59:58.278905 29463 raft_consensus.cc:2454] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 374bd621a26249a2b9e4744235a88c2d in term 2.
I20240502 13:59:58.279173 30936 leader_election.cc:302] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 374bd621a26249a2b9e4744235a88c2d, a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 13:59:58.279348 31392 raft_consensus.cc:2781] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 13:59:58.279443 31392 raft_consensus.cc:686] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [term 2 LEADER]: Becoming Leader. State: Replica: 374bd621a26249a2b9e4744235a88c2d, State: Running, Role: LEADER
I20240502 13:59:58.279598 31392 consensus_queue.cc:237] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } }
I20240502 13:59:58.280874 29188 catalog_manager.cc:5484] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d reported cstate change: term changed from 1 to 2, leader changed from a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) to 374bd621a26249a2b9e4744235a88c2d (127.26.25.135). New cstate: current_term: 2 leader_uuid: "374bd621a26249a2b9e4744235a88c2d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.532342 29463 raft_consensus.cc:1261] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Refusing update from remote peer a18dcdc568884c3d9cf8dda212fc4034: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 13:59:58.532964 31297 consensus_queue.cc:1035] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 13:59:58.590696 29332 raft_consensus.cc:1261] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Refusing update from remote peer 06e5c7647a774d648861526ff1433622: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 13:59:58.591336 31365 consensus_queue.cc:1035] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 13:59:58.608724 29463 raft_consensus.cc:1261] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Refusing update from remote peer 5265d55315f5413bb45818121d568e8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 13:59:58.609232 31347 consensus_queue.cc:1035] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 13:59:58.618592 29332 raft_consensus.cc:1261] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Refusing update from remote peer 0832ab8d65cc47cabeb5e0128a14c151: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 13:59:58.619036 31548 consensus_queue.cc:1035] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 13:59:58.679109 30738 consensus_queue.cc:237] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } }
I20240502 13:59:58.679764 31692 raft_consensus.cc:2932] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } } }
I20240502 13:59:58.681560 29172 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b3fe650164224a218b431f9a52c1283f with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20240502 13:59:58.681807 30607 consensus_queue.cc:237] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } }
I20240502 13:59:58.682384 31297 raft_consensus.cc:2932] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) evicted. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } } }
I20240502 13:59:58.682492 29188 catalog_manager.cc:5484] T b3fe650164224a218b431f9a52c1283f P 0832ab8d65cc47cabeb5e0128a14c151 reported cstate change: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New cstate: current_term: 2 leader_uuid: "0832ab8d65cc47cabeb5e0128a14c151" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.683895 29175 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20240502 13:59:58.684521 29186 catalog_manager.cc:5484] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a18dcdc568884c3d9cf8dda212fc4034 reported cstate change: config changed from index 5 to 7, VOTER a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) evicted. New cstate: current_term: 2 leader_uuid: "a18dcdc568884c3d9cf8dda212fc4034" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.685246 29312 tablet_service.cc:1514] Processing DeleteTablet for tablet b3fe650164224a218b431f9a52c1283f with delete_type TABLET_DATA_TOMBSTONED (TS 506b845ddf0a46ae9fc6a5ce4cafcc8a not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58284
I20240502 13:59:58.685709 31845 tablet_replica.cc:330] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: stopping tablet replica
I20240502 13:59:58.686028 31845 raft_consensus.cc:2229] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 13:59:58.686221 31845 raft_consensus.cc:2258] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 13:59:58.686762 29442 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb with delete_type TABLET_DATA_TOMBSTONED (TS a2d7d0121f2644188e2945b5ec6bf59e not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43736
I20240502 13:59:58.687168 31846 tablet_replica.cc:330] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: stopping tablet replica
I20240502 13:59:58.687350 31845 ts_tablet_manager.cc:1875] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 13:59:58.687462 31846 raft_consensus.cc:2229] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 13:59:58.687618 31846 raft_consensus.cc:2258] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 13:59:58.688472 31846 ts_tablet_manager.cc:1875] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 13:59:58.689302 31845 ts_tablet_manager.cc:1888] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 13:59:58.689388 31845 log.cc:1196] T b3fe650164224a218b431f9a52c1283f P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/b3fe650164224a218b431f9a52c1283f
I20240502 13:59:58.689960 29174 catalog_manager.cc:4830] TS 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111): tablet b3fe650164224a218b431f9a52c1283f (table rebalance_test_table_0 [id=616f782b4b1e4188a882d3553dce32a5]) successfully deleted
I20240502 13:59:58.690335 31846 ts_tablet_manager.cc:1888] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 13:59:58.690403 31846 log.cc:1196] T 4ac7ce8a2d944c86b2aa54c6f746d1eb P a2d7d0121f2644188e2945b5ec6bf59e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/4ac7ce8a2d944c86b2aa54c6f746d1eb
I20240502 13:59:58.690873 29175 catalog_manager.cc:4830] TS a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699): tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb (table rebalance_test_table_3 [id=5bc4c8c9d29c410bbfea486485d41cff]) successfully deleted
I20240502 13:59:58.727694 29332 raft_consensus.cc:1261] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Refusing update from remote peer 06e5c7647a774d648861526ff1433622: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 13:59:58.728163 31744 consensus_queue.cc:1035] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 13:59:58.749511 29463 raft_consensus.cc:1261] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Refusing update from remote peer 0832ab8d65cc47cabeb5e0128a14c151: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 13:59:58.749962 31548 consensus_queue.cc:1035] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 13:59:58.790690 30461 consensus_queue.cc:237] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false } }
I20240502 13:59:58.791523 31742 raft_consensus.cc:2932] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) evicted. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false } } }
I20240502 13:59:58.792814 29174 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet aa5712c33fbf4d8781478a12e76f584a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20240502 13:59:58.793731 29186 catalog_manager.cc:5484] T aa5712c33fbf4d8781478a12e76f584a P 5265d55315f5413bb45818121d568e8b reported cstate change: config changed from index 5 to 7, VOTER a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) evicted. New cstate: current_term: 2 leader_uuid: "5265d55315f5413bb45818121d568e8b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5265d55315f5413bb45818121d568e8b" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 37575 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.795554 30869 consensus_queue.cc:237] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } }
I20240502 13:59:58.796161 31744 raft_consensus.cc:2932] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } } }
I20240502 13:59:58.796293 29332 raft_consensus.cc:1261] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Refusing update from remote peer 374bd621a26249a2b9e4744235a88c2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 13:59:58.796767 31864 consensus_queue.cc:1035] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 13:59:58.797652 29442 tablet_service.cc:1514] Processing DeleteTablet for tablet aa5712c33fbf4d8781478a12e76f584a with delete_type TABLET_DATA_TOMBSTONED (TS a2d7d0121f2644188e2945b5ec6bf59e not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43736
I20240502 13:59:58.797959 29175 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4edf51e537524169b98d62c50947571a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20240502 13:59:58.798105 31846 tablet_replica.cc:330] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: stopping tablet replica
I20240502 13:59:58.798228 31846 raft_consensus.cc:2229] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 13:59:58.798345 31846 raft_consensus.cc:2258] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 13:59:58.798473 29186 catalog_manager.cc:5484] T 4edf51e537524169b98d62c50947571a P 06e5c7647a774d648861526ff1433622 reported cstate change: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New cstate: current_term: 2 leader_uuid: "06e5c7647a774d648861526ff1433622" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.799026 31846 ts_tablet_manager.cc:1875] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 13:59:58.800578 29312 tablet_service.cc:1514] Processing DeleteTablet for tablet 4edf51e537524169b98d62c50947571a with delete_type TABLET_DATA_TOMBSTONED (TS 506b845ddf0a46ae9fc6a5ce4cafcc8a not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58284
I20240502 13:59:58.801065 31846 ts_tablet_manager.cc:1888] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 13:59:58.801136 31846 log.cc:1196] T aa5712c33fbf4d8781478a12e76f584a P a2d7d0121f2644188e2945b5ec6bf59e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/aa5712c33fbf4d8781478a12e76f584a
I20240502 13:59:58.801628 29175 catalog_manager.cc:4830] TS a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699): tablet aa5712c33fbf4d8781478a12e76f584a (table rebalance_test_table_1 [id=f8339f00c8d2464c853e5f878b6a6a7c]) successfully deleted
I20240502 13:59:58.802407 31845 tablet_replica.cc:330] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: stopping tablet replica
I20240502 13:59:58.802532 31845 raft_consensus.cc:2229] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 13:59:58.802610 31845 raft_consensus.cc:2258] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 13:59:58.803251 31845 ts_tablet_manager.cc:1875] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 13:59:58.804153 29463 raft_consensus.cc:1261] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Refusing update from remote peer 374bd621a26249a2b9e4744235a88c2d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 13:59:58.804579 31864 consensus_queue.cc:1035] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 13:59:58.805495 31845 ts_tablet_manager.cc:1888] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 13:59:58.805572 31845 log.cc:1196] T 4edf51e537524169b98d62c50947571a P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/4edf51e537524169b98d62c50947571a
I20240502 13:59:58.806070 29174 catalog_manager.cc:4830] TS 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111): tablet 4edf51e537524169b98d62c50947571a (table rebalance_test_table_3 [id=5bc4c8c9d29c410bbfea486485d41cff]) successfully deleted
I20240502 13:59:58.806993 30738 consensus_queue.cc:237] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } }
I20240502 13:59:58.808071 31548 raft_consensus.cc:2932] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) evicted. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } } }
I20240502 13:59:58.810134 29172 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ad3ff2489bad44a19600cc9dae475806 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20240502 13:59:58.810698 29186 catalog_manager.cc:5484] T ad3ff2489bad44a19600cc9dae475806 P 0832ab8d65cc47cabeb5e0128a14c151 reported cstate change: config changed from index 5 to 7, VOTER a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) evicted. New cstate: current_term: 2 leader_uuid: "0832ab8d65cc47cabeb5e0128a14c151" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0832ab8d65cc47cabeb5e0128a14c151" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 44559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.812978 29442 tablet_service.cc:1514] Processing DeleteTablet for tablet ad3ff2489bad44a19600cc9dae475806 with delete_type TABLET_DATA_TOMBSTONED (TS a2d7d0121f2644188e2945b5ec6bf59e not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43736
I20240502 13:59:58.813201 31846 tablet_replica.cc:330] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: stopping tablet replica
I20240502 13:59:58.813325 31846 raft_consensus.cc:2229] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 13:59:58.813426 31846 raft_consensus.cc:2258] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 13:59:58.814047 31846 ts_tablet_manager.cc:1875] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 13:59:58.815999 31846 ts_tablet_manager.cc:1888] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 13:59:58.816078 31846 log.cc:1196] T ad3ff2489bad44a19600cc9dae475806 P a2d7d0121f2644188e2945b5ec6bf59e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/ad3ff2489bad44a19600cc9dae475806
I20240502 13:59:58.816110 31000 consensus_queue.cc:237] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } }
I20240502 13:59:58.816561 29175 catalog_manager.cc:4830] TS a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699): tablet ad3ff2489bad44a19600cc9dae475806 (table rebalance_test_table_1 [id=f8339f00c8d2464c853e5f878b6a6a7c]) successfully deleted
I20240502 13:59:58.816768 31871 raft_consensus.cc:2932] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } } }
I20240502 13:59:58.817915 29172 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet dad1f80730fa4ca0a26b8ae9f3ab67bf with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20240502 13:59:58.818692 29186 catalog_manager.cc:5484] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 374bd621a26249a2b9e4744235a88c2d reported cstate change: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New cstate: current_term: 2 leader_uuid: "374bd621a26249a2b9e4744235a88c2d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.821066 29312 tablet_service.cc:1514] Processing DeleteTablet for tablet dad1f80730fa4ca0a26b8ae9f3ab67bf with delete_type TABLET_DATA_TOMBSTONED (TS 506b845ddf0a46ae9fc6a5ce4cafcc8a not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58284
I20240502 13:59:58.821537 31845 tablet_replica.cc:330] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: stopping tablet replica
I20240502 13:59:58.821686 31845 raft_consensus.cc:2229] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 13:59:58.821799 31845 raft_consensus.cc:2258] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 13:59:58.822595 31845 ts_tablet_manager.cc:1875] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 13:59:58.824360 31845 ts_tablet_manager.cc:1888] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 13:59:58.824424 31845 log.cc:1196] T dad1f80730fa4ca0a26b8ae9f3ab67bf P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/dad1f80730fa4ca0a26b8ae9f3ab67bf
I20240502 13:59:58.824906 29174 catalog_manager.cc:4830] TS 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111): tablet dad1f80730fa4ca0a26b8ae9f3ab67bf (table rebalance_test_table_4 [id=4af4b17be3404e158177eaa235830e75]) successfully deleted
I20240502 13:59:58.863727 30869 consensus_queue.cc:237] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } }
I20240502 13:59:58.864424 31365 raft_consensus.cc:2932] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } } }
I20240502 13:59:58.865921 29175 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 14bc58294f6a48fd9e4ef5ab2a52afea with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20240502 13:59:58.866747 29186 catalog_manager.cc:5484] T 14bc58294f6a48fd9e4ef5ab2a52afea P 06e5c7647a774d648861526ff1433622 reported cstate change: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New cstate: current_term: 2 leader_uuid: "06e5c7647a774d648861526ff1433622" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "06e5c7647a774d648861526ff1433622" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 35775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.869141 29312 tablet_service.cc:1514] Processing DeleteTablet for tablet 14bc58294f6a48fd9e4ef5ab2a52afea with delete_type TABLET_DATA_TOMBSTONED (TS 506b845ddf0a46ae9fc6a5ce4cafcc8a not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58284
I20240502 13:59:58.869380 31845 tablet_replica.cc:330] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: stopping tablet replica
I20240502 13:59:58.869525 31845 raft_consensus.cc:2229] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 13:59:58.869647 31845 raft_consensus.cc:2258] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 13:59:58.870359 31845 ts_tablet_manager.cc:1875] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 13:59:58.872231 31845 ts_tablet_manager.cc:1888] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 13:59:58.872314 31845 log.cc:1196] T 14bc58294f6a48fd9e4ef5ab2a52afea P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/14bc58294f6a48fd9e4ef5ab2a52afea
I20240502 13:59:58.872829 29174 catalog_manager.cc:4830] TS 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111): tablet 14bc58294f6a48fd9e4ef5ab2a52afea (table rebalance_test_table_2 [id=eed1063799324a01a6b0b87659e9b508]) successfully deleted
I20240502 13:59:58.875239 29332 raft_consensus.cc:1261] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Refusing update from remote peer a18dcdc568884c3d9cf8dda212fc4034: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 13:59:58.875662 31720 consensus_queue.cc:1035] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 13:59:58.883051 30607 consensus_queue.cc:237] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } }
I20240502 13:59:58.883589 31720 raft_consensus.cc:2932] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } } }
I20240502 13:59:58.885030 29175 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8464e256d453478caf17bab1048efd78 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20240502 13:59:58.885764 29186 catalog_manager.cc:5484] T 8464e256d453478caf17bab1048efd78 P a18dcdc568884c3d9cf8dda212fc4034 reported cstate change: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New cstate: current_term: 2 leader_uuid: "a18dcdc568884c3d9cf8dda212fc4034" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a18dcdc568884c3d9cf8dda212fc4034" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38647 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.888029 29312 tablet_service.cc:1514] Processing DeleteTablet for tablet 8464e256d453478caf17bab1048efd78 with delete_type TABLET_DATA_TOMBSTONED (TS 506b845ddf0a46ae9fc6a5ce4cafcc8a not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58284
I20240502 13:59:58.888383 31845 tablet_replica.cc:330] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: stopping tablet replica
I20240502 13:59:58.888518 31845 raft_consensus.cc:2229] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 13:59:58.888630 31845 raft_consensus.cc:2258] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 13:59:58.889214 31845 ts_tablet_manager.cc:1875] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 13:59:58.891216 31845 ts_tablet_manager.cc:1888] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 13:59:58.891295 31845 log.cc:1196] T 8464e256d453478caf17bab1048efd78 P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/8464e256d453478caf17bab1048efd78
I20240502 13:59:58.891795 29174 catalog_manager.cc:4830] TS 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111): tablet 8464e256d453478caf17bab1048efd78 (table rebalance_test_table_4 [id=4af4b17be3404e158177eaa235830e75]) successfully deleted
I20240502 13:59:58.901796 31000 consensus_queue.cc:237] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } }
I20240502 13:59:58.902315 31871 raft_consensus.cc:2932] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) evicted. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } } }
I20240502 13:59:58.903383 29172 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2e17ab6801dd44928a9ff5a5b46d12ed with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20240502 13:59:58.904158 29186 catalog_manager.cc:5484] T 2e17ab6801dd44928a9ff5a5b46d12ed P 374bd621a26249a2b9e4744235a88c2d reported cstate change: config changed from index 5 to 7, VOTER a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) evicted. New cstate: current_term: 2 leader_uuid: "374bd621a26249a2b9e4744235a88c2d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "374bd621a26249a2b9e4744235a88c2d" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 40775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 13:59:58.906191 29442 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e17ab6801dd44928a9ff5a5b46d12ed with delete_type TABLET_DATA_TOMBSTONED (TS a2d7d0121f2644188e2945b5ec6bf59e not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43736
I20240502 13:59:58.906423 31846 tablet_replica.cc:330] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: stopping tablet replica
I20240502 13:59:58.906566 31846 raft_consensus.cc:2229] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 13:59:58.906669 31846 raft_consensus.cc:2258] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 13:59:58.907291 31846 ts_tablet_manager.cc:1875] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 13:59:58.908910 31846 ts_tablet_manager.cc:1888] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 13:59:58.908978 31846 log.cc:1196] T 2e17ab6801dd44928a9ff5a5b46d12ed P a2d7d0121f2644188e2945b5ec6bf59e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/2e17ab6801dd44928a9ff5a5b46d12ed
I20240502 13:59:58.909435 29175 catalog_manager.cc:4830] TS a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699): tablet 2e17ab6801dd44928a9ff5a5b46d12ed (table rebalance_test_table_2 [id=eed1063799324a01a6b0b87659e9b508]) successfully deleted
I20240502 13:59:59.158452 30718 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.160053 30980 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.160907 30444 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.160951 30849 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.162246 29312 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:59.175813 30587 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.199278 29442 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:59.272343 29312 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:59.273336 30849 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.274555 30718 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.276456 30587 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.278959 30980 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.288133 30444 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.296559 29442 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:59.510601 30849 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.511931 30718 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.515236 29312 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:59.519971 30587 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.522023 30444 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.533133 30980 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.539811 29442 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:59.637898 30849 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.640695 30980 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.642850 30718 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.643206 30587 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.644636 30444 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.657213 29442 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:59.658414 29312 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:59.757665 30849 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.761859 29312 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:59.764500 30444 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.768527 30980 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.773697 29442 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:59.775638 30718 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.783097 30587 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
dc2396ac20b549299f014b770066f0c8 | 127.26.25.190:33009 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.26.25.148:38953 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
06e5c7647a774d648861526ff1433622 | 127.26.25.134:35775 | HEALTHY | <none> | 2 | 0
0832ab8d65cc47cabeb5e0128a14c151 | 127.26.25.133:44559 | HEALTHY | <none> | 2 | 0
374bd621a26249a2b9e4744235a88c2d | 127.26.25.135:40775 | HEALTHY | <none> | 2 | 0
506b845ddf0a46ae9fc6a5ce4cafcc8a | 127.26.25.129:35111 | HEALTHY | <none> | 3 | 0
5265d55315f5413bb45818121d568e8b | 127.26.25.131:37575 | HEALTHY | <none> | 2 | 0
a18dcdc568884c3d9cf8dda212fc4034 | 127.26.25.132:38647 | HEALTHY | <none> | 2 | 0
a2d7d0121f2644188e2945b5ec6bf59e | 127.26.25.130:35699 | HEALTHY | <none> | 3 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 7
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 7 server(s) checked
local_ip_for_outbound_sockets | 127.26.25.129 | experimental | 127.26.25.129:35111
local_ip_for_outbound_sockets | 127.26.25.130 | experimental | 127.26.25.130:35699
local_ip_for_outbound_sockets | 127.26.25.131 | experimental | 127.26.25.131:37575
local_ip_for_outbound_sockets | 127.26.25.132 | experimental | 127.26.25.132:38647
local_ip_for_outbound_sockets | 127.26.25.133 | experimental | 127.26.25.133:44559
local_ip_for_outbound_sockets | 127.26.25.134 | experimental | 127.26.25.134:35775
local_ip_for_outbound_sockets | 127.26.25.135 | experimental | 127.26.25.135:40775
never_fsync | true | unsafe,advanced | all 7 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 7 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 7 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 7 server(s) checked
server_dump_info_format | pb | hidden | all 7 server(s) checked
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.26.25.129:35111
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.26.25.130:35699
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.26.25.131:37575
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.26.25.132:38647
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.26.25.133:44559
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.26.25.134:35775
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.26.25.135:40775
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.26.25.148:38953 | all 7 server(s) checked
time_source | builtin | all 7 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 8 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 3
Maximum | 3
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 7
Tables | 6
Tablets | 16
Replicas | 16
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 13:59:59.814152 26726 log_verifier.cc:126] Checking tablet 0b4f51ba273c44f5805d3744caa4aa3d
I20240502 13:59:59.824827 26726 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 0b4f51ba273c44f5805d3744caa4aa3d
I20240502 13:59:59.824928 26726 log_verifier.cc:126] Checking tablet 14bc58294f6a48fd9e4ef5ab2a52afea
I20240502 13:59:59.833098 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 14bc58294f6a48fd9e4ef5ab2a52afea
I20240502 13:59:59.833149 26726 log_verifier.cc:126] Checking tablet 2e17ab6801dd44928a9ff5a5b46d12ed
I20240502 13:59:59.841112 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 2e17ab6801dd44928a9ff5a5b46d12ed
I20240502 13:59:59.841168 26726 log_verifier.cc:126] Checking tablet 418124b2a16a49a8aaf0e50473b9bf50
I20240502 13:59:59.847904 26726 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 418124b2a16a49a8aaf0e50473b9bf50
I20240502 13:59:59.847954 26726 log_verifier.cc:126] Checking tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb
I20240502 13:59:59.855993 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb
I20240502 13:59:59.856053 26726 log_verifier.cc:126] Checking tablet 4edf51e537524169b98d62c50947571a
I20240502 13:59:59.864040 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 4edf51e537524169b98d62c50947571a
I20240502 13:59:59.864094 26726 log_verifier.cc:126] Checking tablet 576317299e6a4c15a742338c67292deb
I20240502 13:59:59.872200 26726 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 576317299e6a4c15a742338c67292deb
I20240502 13:59:59.872256 26726 log_verifier.cc:126] Checking tablet 715eca20d6774ced9b9dba8d831bf6ac
I20240502 13:59:59.880357 26726 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 715eca20d6774ced9b9dba8d831bf6ac
I20240502 13:59:59.880411 26726 log_verifier.cc:126] Checking tablet 8464e256d453478caf17bab1048efd78
I20240502 13:59:59.888790 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 8464e256d453478caf17bab1048efd78
I20240502 13:59:59.888857 26726 log_verifier.cc:126] Checking tablet aa5712c33fbf4d8781478a12e76f584a
I20240502 13:59:59.897176 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet aa5712c33fbf4d8781478a12e76f584a
I20240502 13:59:59.897229 26726 log_verifier.cc:126] Checking tablet ad3ff2489bad44a19600cc9dae475806
I20240502 13:59:59.905164 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet ad3ff2489bad44a19600cc9dae475806
I20240502 13:59:59.905217 26726 log_verifier.cc:126] Checking tablet b3fe650164224a218b431f9a52c1283f
I20240502 13:59:59.913767 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet b3fe650164224a218b431f9a52c1283f
I20240502 13:59:59.913822 26726 log_verifier.cc:126] Checking tablet c714a72f483440ec9423d736b2a28786
I20240502 13:59:59.921635 26726 log_verifier.cc:177] Verified matching terms for 3 ops in tablet c714a72f483440ec9423d736b2a28786
I20240502 13:59:59.921691 26726 log_verifier.cc:126] Checking tablet d4d2fe5bc85945659a881f83eff3fc23
I20240502 13:59:59.929710 26726 log_verifier.cc:177] Verified matching terms for 3 ops in tablet d4d2fe5bc85945659a881f83eff3fc23
I20240502 13:59:59.929767 26726 log_verifier.cc:126] Checking tablet dad1f80730fa4ca0a26b8ae9f3ab67bf
I20240502 13:59:59.938190 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet dad1f80730fa4ca0a26b8ae9f3ab67bf
I20240502 13:59:59.938256 26726 log_verifier.cc:126] Checking tablet dd53513bf2744b66b3da0d7548a04b38
I20240502 13:59:59.946121 26726 log_verifier.cc:177] Verified matching terms for 3 ops in tablet dd53513bf2744b66b3da0d7548a04b38
I20240502 13:59:59.948205 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.136:0
--local_ip_for_outbound_sockets=127.26.25.136
--webserver_interface=127.26.25.136
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--builtin_ntp_servers=127.26.25.148:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:00:00.053236 32131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:00.053421 32131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:00.053465 32131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:00.056463 32131 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:00.056522 32131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:00.056605 32131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.136
I20240502 14:00:00.059978 32131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.26.25.136
--webserver_port=0
--tserver_master_addrs=127.26.25.190:33009
--never_fsync=true
--heap_profile_path=/tmp/kudu.32131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.136
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:00.060737 32131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:00.061751 32131 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:00.069903 32137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:00.070636 32139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:00.071385 32136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:00.072028 32131 server_base.cc:1034] running on GCE node
I20240502 14:00:00.072357 32131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:00.072835 32131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:00.073992 32131 hybrid_clock.cc:648] HybridClock initialized: now 1714658400073975 us; error 39 us; skew 500 ppm
I20240502 14:00:00.075898 32131 webserver.cc:460] Webserver started at http://127.26.25.136:34017/ using document root <none> and password file <none>
I20240502 14:00:00.076423 32131 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:00.076493 32131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:00.076682 32131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:00.078223 32131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "52e5f486c118494e8ac22a14f6d60889"
format_stamp: "Formatted at 2024-05-02 14:00:00 on dist-test-slave-sldb"
I20240502 14:00:00.078665 32131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "52e5f486c118494e8ac22a14f6d60889"
format_stamp: "Formatted at 2024-05-02 14:00:00 on dist-test-slave-sldb"
I20240502 14:00:00.081869 32131 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20240502 14:00:00.083789 32145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:00.084738 32131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 14:00:00.084834 32131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "52e5f486c118494e8ac22a14f6d60889"
format_stamp: "Formatted at 2024-05-02 14:00:00 on dist-test-slave-sldb"
I20240502 14:00:00.084925 32131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:00.100986 32131 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:00.101562 32131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:00.101696 32131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:00.102164 32131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:00.102948 32131 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:00.102993 32131 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:00.103034 32131 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:00.103062 32131 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:00.111119 32131 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.136:42587
I20240502 14:00:00.111145 32258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.136:42587 every 8 connection(s)
I20240502 14:00:00.112026 32131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 14:00:00.113441 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 32131
I20240502 14:00:00.113548 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 14:00:00.120507 32259 heartbeater.cc:347] Connected to a master server at 127.26.25.190:33009
I20240502 14:00:00.120712 32259 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:00.121167 32259 heartbeater.cc:511] Master 127.26.25.190:33009 requested a full tablet report, sending...
I20240502 14:00:00.122174 29188 ts_manager.cc:194] Registered new tserver with Master: 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136:42587)
I20240502 14:00:00.122987 29188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.136:38149
I20240502 14:00:00.300477 30849 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:00.302038 29312 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:00.302752 30718 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:00.303195 30980 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:00.308586 29442 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:00.317276 30587 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:00.319316 32193 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:00.322373 30444 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:00.445343 30849 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:00.445880 30980 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:00.452029 29312 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:00.452283 30587 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:00.457803 30718 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:00.466195 29442 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:00.476501 32193 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:00.479528 30444 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:00.508350 29463 consensus_queue.cc:237] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: true } }
I20240502 14:00:00.509300 32352 raft_consensus.cc:2932] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: true } } }
I20240502 14:00:00.512502 29181 catalog_manager.cc:5484] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: config changed from index -1 to 4, NON_VOTER 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:00.514068 29396 consensus_peers.cc:485] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e -> Peer 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136:42587): Couldn't send request to peer 52e5f486c118494e8ac22a14f6d60889. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd53513bf2744b66b3da0d7548a04b38. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:00.518669 29332 consensus_queue.cc:237] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: true } }
I20240502 14:00:00.519517 32365 raft_consensus.cc:2932] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: true } } }
I20240502 14:00:00.522367 29181 catalog_manager.cc:5484] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: config changed from index -1 to 4, NON_VOTER 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:00.523893 29264 consensus_peers.cc:485] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a -> Peer 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136:42587): Couldn't send request to peer 52e5f486c118494e8ac22a14f6d60889. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b4f51ba273c44f5805d3744caa4aa3d. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:01.062734 32399 ts_tablet_manager.cc:902] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889: Initiating tablet copy from peer 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 14:00:01.063412 32399 tablet_copy_client.cc:308] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:35111
I20240502 14:00:01.067042 29352 tablet_copy_service.cc:140] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Received BeginTabletCopySession request for tablet 0b4f51ba273c44f5805d3744caa4aa3d from peer 52e5f486c118494e8ac22a14f6d60889 ({username='slave'} at 127.26.25.136:35783)
I20240502 14:00:01.067142 29352 tablet_copy_service.cc:161] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Beginning new tablet copy session on tablet 0b4f51ba273c44f5805d3744caa4aa3d from peer 52e5f486c118494e8ac22a14f6d60889 at {username='slave'} at 127.26.25.136:35783: session id = 52e5f486c118494e8ac22a14f6d60889-0b4f51ba273c44f5805d3744caa4aa3d
I20240502 14:00:01.067876 29352 tablet_copy_source_session.cc:205] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:01.070899 32399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b4f51ba273c44f5805d3744caa4aa3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:01.075640 32399 tablet_copy_client.cc:783] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:01.075841 32399 tablet_copy_client.cc:647] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:01.077208 32399 tablet_copy_client.cc:523] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:01.078590 32399 tablet_bootstrap.cc:492] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889: Bootstrap starting.
I20240502 14:00:01.083142 32399 log.cc:824] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:01.093009 32407 ts_tablet_manager.cc:902] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889: Initiating tablet copy from peer a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699)
I20240502 14:00:01.093525 32407 tablet_copy_client.cc:308] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35699
I20240502 14:00:01.097257 29484 tablet_copy_service.cc:140] P a2d7d0121f2644188e2945b5ec6bf59e: Received BeginTabletCopySession request for tablet dd53513bf2744b66b3da0d7548a04b38 from peer 52e5f486c118494e8ac22a14f6d60889 ({username='slave'} at 127.26.25.136:43409)
I20240502 14:00:01.097353 29484 tablet_copy_service.cc:161] P a2d7d0121f2644188e2945b5ec6bf59e: Beginning new tablet copy session on tablet dd53513bf2744b66b3da0d7548a04b38 from peer 52e5f486c118494e8ac22a14f6d60889 at {username='slave'} at 127.26.25.136:43409: session id = 52e5f486c118494e8ac22a14f6d60889-dd53513bf2744b66b3da0d7548a04b38
I20240502 14:00:01.097985 29484 tablet_copy_source_session.cc:205] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:01.098740 32407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd53513bf2744b66b3da0d7548a04b38. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:01.100400 32407 tablet_copy_client.cc:783] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:01.100632 32407 tablet_copy_client.cc:647] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:01.102164 32407 tablet_copy_client.cc:523] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:01.103055 32407 tablet_bootstrap.cc:492] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889: Bootstrap starting.
I20240502 14:00:01.103857 32399 tablet_bootstrap.cc:492] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:01.104133 32399 tablet_bootstrap.cc:492] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889: Bootstrap complete.
I20240502 14:00:01.104390 32399 ts_tablet_manager.cc:1367] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889: Time spent bootstrapping tablet: real 0.026s user 0.023s sys 0.004s
I20240502 14:00:01.106216 32399 raft_consensus.cc:348] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: true } }
I20240502 14:00:01.106441 32399 raft_consensus.cc:729] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52e5f486c118494e8ac22a14f6d60889, State: Initialized, Role: LEARNER
I20240502 14:00:01.106758 32399 consensus_queue.cc:260] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: true } }
I20240502 14:00:01.107370 32259 heartbeater.cc:503] Master 127.26.25.190:33009 was elected leader, sending a full tablet report...
I20240502 14:00:01.107563 32399 ts_tablet_manager.cc:1398] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 14:00:01.108173 29352 tablet_copy_service.cc:342] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Request end of tablet copy session 52e5f486c118494e8ac22a14f6d60889-0b4f51ba273c44f5805d3744caa4aa3d received from {username='slave'} at 127.26.25.136:35783
I20240502 14:00:01.108254 29352 tablet_copy_service.cc:433] P 506b845ddf0a46ae9fc6a5ce4cafcc8a: ending tablet copy session 52e5f486c118494e8ac22a14f6d60889-0b4f51ba273c44f5805d3744caa4aa3d on tablet 0b4f51ba273c44f5805d3744caa4aa3d with peer 52e5f486c118494e8ac22a14f6d60889
I20240502 14:00:01.125589 32407 tablet_bootstrap.cc:492] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:01.125814 32407 tablet_bootstrap.cc:492] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889: Bootstrap complete.
I20240502 14:00:01.125947 32407 ts_tablet_manager.cc:1367] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889: Time spent bootstrapping tablet: real 0.023s user 0.014s sys 0.007s
I20240502 14:00:01.126367 32407 raft_consensus.cc:348] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: true } }
I20240502 14:00:01.126459 32407 raft_consensus.cc:729] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52e5f486c118494e8ac22a14f6d60889, State: Initialized, Role: LEARNER
I20240502 14:00:01.126542 32407 consensus_queue.cc:260] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: true } }
I20240502 14:00:01.126770 32407 ts_tablet_manager.cc:1398] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:01.127297 29484 tablet_copy_service.cc:342] P a2d7d0121f2644188e2945b5ec6bf59e: Request end of tablet copy session 52e5f486c118494e8ac22a14f6d60889-dd53513bf2744b66b3da0d7548a04b38 received from {username='slave'} at 127.26.25.136:43409
I20240502 14:00:01.127398 29484 tablet_copy_service.cc:433] P a2d7d0121f2644188e2945b5ec6bf59e: ending tablet copy session 52e5f486c118494e8ac22a14f6d60889-dd53513bf2744b66b3da0d7548a04b38 on tablet dd53513bf2744b66b3da0d7548a04b38 with peer 52e5f486c118494e8ac22a14f6d60889
I20240502 14:00:01.528386 32213 raft_consensus.cc:1203] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:01.671453 32213 raft_consensus.cc:1203] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:02.052217 32470 raft_consensus.cc:1050] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: attempting to promote NON_VOTER 52e5f486c118494e8ac22a14f6d60889 to VOTER
I20240502 14:00:02.052614 32470 consensus_queue.cc:237] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } }
I20240502 14:00:02.053771 32213 raft_consensus.cc:1261] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [term 1 LEARNER]: Refusing update from remote peer a2d7d0121f2644188e2945b5ec6bf59e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:02.054181 32470 consensus_queue.cc:1035] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [LEADER]: Connected to new peer: Peer: permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:02.056697 32469 raft_consensus.cc:2932] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } } }
I20240502 14:00:02.057178 32213 raft_consensus.cc:2932] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } } }
I20240502 14:00:02.058948 29179 catalog_manager.cc:5484] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e reported cstate change: config changed from index 4 to 5, 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:02.201400 32479 raft_consensus.cc:1050] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: attempting to promote NON_VOTER 52e5f486c118494e8ac22a14f6d60889 to VOTER
I20240502 14:00:02.201753 32479 consensus_queue.cc:237] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } }
I20240502 14:00:02.202807 32213 raft_consensus.cc:1261] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [term 1 LEARNER]: Refusing update from remote peer 506b845ddf0a46ae9fc6a5ce4cafcc8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:02.203207 32479 consensus_queue.cc:1035] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:02.204970 32478 raft_consensus.cc:2932] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } } }
I20240502 14:00:02.205387 32213 raft_consensus.cc:2932] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } } }
I20240502 14:00:02.207549 29181 catalog_manager.cc:5484] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a reported cstate change: config changed from index 4 to 5, 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:02.226629 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b4f51ba273c44f5805d3744caa4aa3d"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55568
I20240502 14:00:02.226763 29332 raft_consensus.cc:595] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:02.236644 29463 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dd53513bf2744b66b3da0d7548a04b38"
dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43752
I20240502 14:00:02.236778 29463 raft_consensus.cc:595] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:02.447803 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b4f51ba273c44f5805d3744caa4aa3d"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55584
I20240502 14:00:02.448002 29332 raft_consensus.cc:595] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:02.457784 29463 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dd53513bf2744b66b3da0d7548a04b38"
dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43778
I20240502 14:00:02.457918 29463 raft_consensus.cc:595] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:02.505342 32470 raft_consensus.cc:979] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: : Instructing follower 52e5f486c118494e8ac22a14f6d60889 to start an election
I20240502 14:00:02.505451 32470 raft_consensus.cc:1067] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Signalling peer 52e5f486c118494e8ac22a14f6d60889 to start an election
I20240502 14:00:02.505924 32213 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dd53513bf2744b66b3da0d7548a04b38"
dest_uuid: "52e5f486c118494e8ac22a14f6d60889"
from {username='slave'} at 127.26.25.130:39041
I20240502 14:00:02.506117 32213 raft_consensus.cc:482] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:02.506224 32213 raft_consensus.cc:3037] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:02.507057 32213 raft_consensus.cc:504] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } }
I20240502 14:00:02.507911 32213 leader_election.cc:288] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [CANDIDATE]: Term 2 election: Requested vote from peers a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699)
I20240502 14:00:02.508285 29463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd53513bf2744b66b3da0d7548a04b38" candidate_uuid: "52e5f486c118494e8ac22a14f6d60889" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "a2d7d0121f2644188e2945b5ec6bf59e"
I20240502 14:00:02.508399 29463 raft_consensus.cc:3032] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:02.508447 29463 raft_consensus.cc:729] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2d7d0121f2644188e2945b5ec6bf59e, State: Running, Role: LEADER
I20240502 14:00:02.508597 29463 consensus_queue.cc:260] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } }
I20240502 14:00:02.508724 29463 raft_consensus.cc:3037] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:02.509454 29463 raft_consensus.cc:2454] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52e5f486c118494e8ac22a14f6d60889 in term 2.
I20240502 14:00:02.509809 32149 leader_election.cc:302] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 52e5f486c118494e8ac22a14f6d60889, a2d7d0121f2644188e2945b5ec6bf59e; no voters:
I20240502 14:00:02.510027 32413 raft_consensus.cc:2781] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:02.510202 32413 raft_consensus.cc:686] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [term 2 LEADER]: Becoming Leader. State: Replica: 52e5f486c118494e8ac22a14f6d60889, State: Running, Role: LEADER
I20240502 14:00:02.510452 32413 consensus_queue.cc:237] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } }
I20240502 14:00:02.512315 29179 catalog_manager.cc:5484] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 reported cstate change: term changed from 1 to 2, leader changed from a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) to 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136). New cstate: current_term: 2 leader_uuid: "52e5f486c118494e8ac22a14f6d60889" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20240502 14:00:02.517967 29172 catalog_manager.cc:5014] ChangeConfig:REMOVE_PEER RPC for tablet dd53513bf2744b66b3da0d7548a04b38 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136:42587) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20240502 14:00:02.668758 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b4f51ba273c44f5805d3744caa4aa3d"
dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55600
I20240502 14:00:02.668898 29332 raft_consensus.cc:595] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:02.711133 32540 raft_consensus.cc:979] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: : Instructing follower 52e5f486c118494e8ac22a14f6d60889 to start an election
I20240502 14:00:02.711230 32540 raft_consensus.cc:1067] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Signalling peer 52e5f486c118494e8ac22a14f6d60889 to start an election
I20240502 14:00:02.711652 32213 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b4f51ba273c44f5805d3744caa4aa3d"
dest_uuid: "52e5f486c118494e8ac22a14f6d60889"
from {username='slave'} at 127.26.25.129:41469
I20240502 14:00:02.711791 32213 raft_consensus.cc:482] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:02.711885 32213 raft_consensus.cc:3037] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:02.712733 32213 raft_consensus.cc:504] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } }
I20240502 14:00:02.713099 32213 leader_election.cc:288] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [CANDIDATE]: Term 2 election: Requested vote from peers 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111)
I20240502 14:00:02.713459 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b4f51ba273c44f5805d3744caa4aa3d" candidate_uuid: "52e5f486c118494e8ac22a14f6d60889" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a"
I20240502 14:00:02.713562 29332 raft_consensus.cc:3032] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:02.713608 29332 raft_consensus.cc:729] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506b845ddf0a46ae9fc6a5ce4cafcc8a, State: Running, Role: LEADER
I20240502 14:00:02.713743 29332 consensus_queue.cc:260] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } }
I20240502 14:00:02.713861 29332 raft_consensus.cc:3037] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:02.714584 29332 raft_consensus.cc:2454] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52e5f486c118494e8ac22a14f6d60889 in term 2.
I20240502 14:00:02.714913 32148 leader_election.cc:302] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 506b845ddf0a46ae9fc6a5ce4cafcc8a, 52e5f486c118494e8ac22a14f6d60889; no voters:
I20240502 14:00:02.715088 32413 raft_consensus.cc:2781] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:02.715196 32413 raft_consensus.cc:686] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [term 2 LEADER]: Becoming Leader. State: Replica: 52e5f486c118494e8ac22a14f6d60889, State: Running, Role: LEADER
I20240502 14:00:02.715337 32413 consensus_queue.cc:237] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } }
I20240502 14:00:02.716760 29179 catalog_manager.cc:5484] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 reported cstate change: term changed from 1 to 2, leader changed from 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) to 52e5f486c118494e8ac22a14f6d60889 (127.26.25.136). New cstate: current_term: 2 leader_uuid: "52e5f486c118494e8ac22a14f6d60889" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:02.966166 29463 raft_consensus.cc:1261] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Refusing update from remote peer 52e5f486c118494e8ac22a14f6d60889: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:00:02.966683 32413 consensus_queue.cc:1035] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2d7d0121f2644188e2945b5ec6bf59e" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35699 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:03.151340 32213 consensus_queue.cc:237] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } }
I20240502 14:00:03.152051 32413 raft_consensus.cc:2932] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) evicted. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } } }
I20240502 14:00:03.153312 29172 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet dd53513bf2744b66b3da0d7548a04b38 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20240502 14:00:03.154253 29181 catalog_manager.cc:5484] T dd53513bf2744b66b3da0d7548a04b38 P 52e5f486c118494e8ac22a14f6d60889 reported cstate change: config changed from index 5 to 7, VOTER a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130) evicted. New cstate: current_term: 2 leader_uuid: "52e5f486c118494e8ac22a14f6d60889" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:03.156594 29442 tablet_service.cc:1514] Processing DeleteTablet for tablet dd53513bf2744b66b3da0d7548a04b38 with delete_type TABLET_DATA_TOMBSTONED (TS a2d7d0121f2644188e2945b5ec6bf59e not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43736
I20240502 14:00:03.156940 32564 tablet_replica.cc:330] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: stopping tablet replica
I20240502 14:00:03.157063 32564 raft_consensus.cc:2229] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:03.157161 32564 raft_consensus.cc:2258] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:03.158394 32564 ts_tablet_manager.cc:1875] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:03.160169 32564 ts_tablet_manager.cc:1888] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 14:00:03.160234 32564 log.cc:1196] T dd53513bf2744b66b3da0d7548a04b38 P a2d7d0121f2644188e2945b5ec6bf59e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/dd53513bf2744b66b3da0d7548a04b38
I20240502 14:00:03.160696 29175 catalog_manager.cc:4830] TS a2d7d0121f2644188e2945b5ec6bf59e (127.26.25.130:35699): tablet dd53513bf2744b66b3da0d7548a04b38 (table rebalance_test_table_3 [id=5bc4c8c9d29c410bbfea486485d41cff]) successfully deleted
I20240502 14:00:03.236088 29332 raft_consensus.cc:1261] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Refusing update from remote peer 52e5f486c118494e8ac22a14f6d60889: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:00:03.236586 32522 consensus_queue.cc:1035] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506b845ddf0a46ae9fc6a5ce4cafcc8a" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 35111 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:03.338831 32213 consensus_queue.cc:237] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } }
I20240502 14:00:03.339448 32522 raft_consensus.cc:2932] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } } }
I20240502 14:00:03.340698 29172 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0b4f51ba273c44f5805d3744caa4aa3d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20240502 14:00:03.341536 29181 catalog_manager.cc:5484] T 0b4f51ba273c44f5805d3744caa4aa3d P 52e5f486c118494e8ac22a14f6d60889 reported cstate change: config changed from index 5 to 7, VOTER 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129) evicted. New cstate: current_term: 2 leader_uuid: "52e5f486c118494e8ac22a14f6d60889" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "52e5f486c118494e8ac22a14f6d60889" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 42587 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:03.343812 29312 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b4f51ba273c44f5805d3744caa4aa3d with delete_type TABLET_DATA_TOMBSTONED (TS 506b845ddf0a46ae9fc6a5ce4cafcc8a not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58284
I20240502 14:00:03.344190 32583 tablet_replica.cc:330] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: stopping tablet replica
I20240502 14:00:03.344318 32583 raft_consensus.cc:2229] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:03.344431 32583 raft_consensus.cc:2258] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:03.345777 32583 ts_tablet_manager.cc:1875] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:03.347625 32583 ts_tablet_manager.cc:1888] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 14:00:03.347702 32583 log.cc:1196] T 0b4f51ba273c44f5805d3744caa4aa3d P 506b845ddf0a46ae9fc6a5ce4cafcc8a: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/0b4f51ba273c44f5805d3744caa4aa3d
I20240502 14:00:03.348228 29174 catalog_manager.cc:4830] TS 506b845ddf0a46ae9fc6a5ce4cafcc8a (127.26.25.129:35111): tablet 0b4f51ba273c44f5805d3744caa4aa3d (table rebalance_test_table_2 [id=eed1063799324a01a6b0b87659e9b508]) successfully deleted
I20240502 14:00:03.576763 30849 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.578676 29312 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.585736 30718 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.586954 30587 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.587591 32193 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.601466 30444 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.613116 29442 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.616354 30980 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.690567 30849 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.692987 29312 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.697623 32193 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.698096 30444 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.713548 30980 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.714826 30718 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.719362 29442 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.722270 30587 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.813313 30849 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.820960 30587 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.820963 30444 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.821852 30718 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.828039 29312 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.828958 30980 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.829314 32193 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:03.858261 29442 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
dc2396ac20b549299f014b770066f0c8 | 127.26.25.190:33009 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.26.25.148:38953 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
06e5c7647a774d648861526ff1433622 | 127.26.25.134:35775 | HEALTHY | <none> | 2 | 0
0832ab8d65cc47cabeb5e0128a14c151 | 127.26.25.133:44559 | HEALTHY | <none> | 2 | 0
374bd621a26249a2b9e4744235a88c2d | 127.26.25.135:40775 | HEALTHY | <none> | 2 | 0
506b845ddf0a46ae9fc6a5ce4cafcc8a | 127.26.25.129:35111 | HEALTHY | <none> | 2 | 0
5265d55315f5413bb45818121d568e8b | 127.26.25.131:37575 | HEALTHY | <none> | 2 | 0
52e5f486c118494e8ac22a14f6d60889 | 127.26.25.136:42587 | HEALTHY | <none> | 2 | 0
a18dcdc568884c3d9cf8dda212fc4034 | 127.26.25.132:38647 | HEALTHY | <none> | 2 | 0
a2d7d0121f2644188e2945b5ec6bf59e | 127.26.25.130:35699 | HEALTHY | <none> | 2 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.26.25.129 | experimental | 127.26.25.129:35111
local_ip_for_outbound_sockets | 127.26.25.130 | experimental | 127.26.25.130:35699
local_ip_for_outbound_sockets | 127.26.25.131 | experimental | 127.26.25.131:37575
local_ip_for_outbound_sockets | 127.26.25.132 | experimental | 127.26.25.132:38647
local_ip_for_outbound_sockets | 127.26.25.133 | experimental | 127.26.25.133:44559
local_ip_for_outbound_sockets | 127.26.25.134 | experimental | 127.26.25.134:35775
local_ip_for_outbound_sockets | 127.26.25.135 | experimental | 127.26.25.135:40775
local_ip_for_outbound_sockets | 127.26.25.136 | experimental | 127.26.25.136:42587
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.26.25.129:35111
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.26.25.130:35699
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.26.25.131:37575
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.26.25.132:38647
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.26.25.133:44559
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.26.25.134:35775
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.26.25.135:40775
server_dump_info_path | /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalanceParamTest.Rebalance_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.26.25.136:42587
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.26.25.148:38953 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 2
Maximum | 2
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 6
Tablets | 16
Replicas | 16
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 14:00:03.877604 26726 log_verifier.cc:126] Checking tablet 0b4f51ba273c44f5805d3744caa4aa3d
I20240502 14:00:03.886725 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 0b4f51ba273c44f5805d3744caa4aa3d
I20240502 14:00:03.886785 26726 log_verifier.cc:126] Checking tablet 14bc58294f6a48fd9e4ef5ab2a52afea
I20240502 14:00:03.892868 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 14bc58294f6a48fd9e4ef5ab2a52afea
I20240502 14:00:03.892922 26726 log_verifier.cc:126] Checking tablet 2e17ab6801dd44928a9ff5a5b46d12ed
I20240502 14:00:03.898794 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 2e17ab6801dd44928a9ff5a5b46d12ed
I20240502 14:00:03.898850 26726 log_verifier.cc:126] Checking tablet 418124b2a16a49a8aaf0e50473b9bf50
I20240502 14:00:03.903242 26726 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 418124b2a16a49a8aaf0e50473b9bf50
I20240502 14:00:03.903288 26726 log_verifier.cc:126] Checking tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb
I20240502 14:00:03.908869 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 4ac7ce8a2d944c86b2aa54c6f746d1eb
I20240502 14:00:03.908918 26726 log_verifier.cc:126] Checking tablet 4edf51e537524169b98d62c50947571a
I20240502 14:00:03.914661 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 4edf51e537524169b98d62c50947571a
I20240502 14:00:03.914708 26726 log_verifier.cc:126] Checking tablet 576317299e6a4c15a742338c67292deb
I20240502 14:00:03.920243 26726 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 576317299e6a4c15a742338c67292deb
I20240502 14:00:03.920289 26726 log_verifier.cc:126] Checking tablet 715eca20d6774ced9b9dba8d831bf6ac
I20240502 14:00:03.925727 26726 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 715eca20d6774ced9b9dba8d831bf6ac
I20240502 14:00:03.925781 26726 log_verifier.cc:126] Checking tablet 8464e256d453478caf17bab1048efd78
I20240502 14:00:03.931504 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 8464e256d453478caf17bab1048efd78
I20240502 14:00:03.931555 26726 log_verifier.cc:126] Checking tablet aa5712c33fbf4d8781478a12e76f584a
I20240502 14:00:03.937369 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet aa5712c33fbf4d8781478a12e76f584a
I20240502 14:00:03.937418 26726 log_verifier.cc:126] Checking tablet ad3ff2489bad44a19600cc9dae475806
I20240502 14:00:03.942975 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet ad3ff2489bad44a19600cc9dae475806
I20240502 14:00:03.943033 26726 log_verifier.cc:126] Checking tablet b3fe650164224a218b431f9a52c1283f
I20240502 14:00:03.948731 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet b3fe650164224a218b431f9a52c1283f
I20240502 14:00:03.948779 26726 log_verifier.cc:126] Checking tablet c714a72f483440ec9423d736b2a28786
I20240502 14:00:03.954259 26726 log_verifier.cc:177] Verified matching terms for 3 ops in tablet c714a72f483440ec9423d736b2a28786
I20240502 14:00:03.954308 26726 log_verifier.cc:126] Checking tablet d4d2fe5bc85945659a881f83eff3fc23
I20240502 14:00:03.959661 26726 log_verifier.cc:177] Verified matching terms for 3 ops in tablet d4d2fe5bc85945659a881f83eff3fc23
I20240502 14:00:03.959707 26726 log_verifier.cc:126] Checking tablet dad1f80730fa4ca0a26b8ae9f3ab67bf
I20240502 14:00:03.965453 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet dad1f80730fa4ca0a26b8ae9f3ab67bf
I20240502 14:00:03.965500 26726 log_verifier.cc:126] Checking tablet dd53513bf2744b66b3da0d7548a04b38
I20240502 14:00:03.973850 26726 log_verifier.cc:177] Verified matching terms for 7 ops in tablet dd53513bf2744b66b3da0d7548a04b38
I20240502 14:00:03.974314 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 29249
I20240502 14:00:03.981649 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 29381
I20240502 14:00:03.989264 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 30387
I20240502 14:00:03.994858 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 30520
I20240502 14:00:04.000674 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 30656
I20240502 14:00:04.007373 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 30787
I20240502 14:00:04.012904 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 30918
I20240502 14:00:04.018599 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 32131
I20240502 14:00:04.024261 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 29157
2024-05-02T14:00:04Z chronyd exiting
[ OK ] RebalanceParamTest.Rebalance/1 (15928 ms)
[----------] 1 test from RebalanceParamTest (15928 ms total)
[----------] 1 test from DDLDuringRebalancingTest
[ RUN ] DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing/1
I20240502 14:00:04.063589 26726 test_util.cc:268] Using random seed: 85317982
I20240502 14:00:04.064397 26726 ts_itest-base.cc:112] Starting cluster with:
I20240502 14:00:04.064436 26726 ts_itest-base.cc:113] --------------
I20240502 14:00:04.064447 26726 ts_itest-base.cc:114] 8 tablet servers
I20240502 14:00:04.064457 26726 ts_itest-base.cc:115] 3 replicas per TS
I20240502 14:00:04.064468 26726 ts_itest-base.cc:116] --------------
2024-05-02T14:00:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T14:00:04Z Disabled control of system clock
I20240502 14:00:04.072799 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.25.190:42179
--webserver_interface=127.26.25.190
--webserver_port=0
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.25.190:42179
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240502 14:00:04.177596 32703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:04.177834 32703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:04.177906 32703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:04.180882 32703 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:04.180948 32703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 14:00:04.180989 32703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:04.181034 32703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 14:00:04.181073 32703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 14:00:04.184623 32703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.26.25.190:42179
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.25.190:42179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.26.25.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.32703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:04.185459 32703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:04.186547 32703 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:04.193969 32711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:04.194473 32709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:04.195114 32708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:04.196290 32703 server_base.cc:1034] running on GCE node
I20240502 14:00:04.196839 32703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:04.197757 32703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:04.198912 32703 hybrid_clock.cc:648] HybridClock initialized: now 1714658404198903 us; error 29 us; skew 500 ppm
I20240502 14:00:04.200992 32703 webserver.cc:460] Webserver started at http://127.26.25.190:41785/ using document root <none> and password file <none>
I20240502 14:00:04.201520 32703 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:04.201581 32703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:04.201750 32703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:04.203317 32703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "1ab7f53ff5194e848de277ee27020cc8"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.203764 32703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "1ab7f53ff5194e848de277ee27020cc8"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.206985 32703 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240502 14:00:04.209146 32717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:04.210127 32703 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:04.210259 32703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
uuid: "1ab7f53ff5194e848de277ee27020cc8"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.210348 32703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:04.226239 32703 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:04.226810 32703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:04.226970 32703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:04.233110 32703 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.190:42179
I20240502 14:00:04.233130 301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.190:42179 every 8 connection(s)
I20240502 14:00:04.233990 32703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 14:00:04.236394 302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:04.237672 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 32703
I20240502 14:00:04.237813 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 14:00:04.242164 302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Bootstrap starting.
I20240502 14:00:04.244995 302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:04.245930 302 log.cc:824] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:04.247845 302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: No bootstrap required, opened a new log
I20240502 14:00:04.251291 302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab7f53ff5194e848de277ee27020cc8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42179 } }
I20240502 14:00:04.251474 302 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:04.251556 302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab7f53ff5194e848de277ee27020cc8, State: Initialized, Role: FOLLOWER
I20240502 14:00:04.251971 302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab7f53ff5194e848de277ee27020cc8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42179 } }
I20240502 14:00:04.252099 302 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:04.252159 302 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:04.252269 302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:04.252998 302 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab7f53ff5194e848de277ee27020cc8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42179 } }
I20240502 14:00:04.253340 302 leader_election.cc:302] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab7f53ff5194e848de277ee27020cc8; no voters:
I20240502 14:00:04.253613 302 leader_election.cc:288] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:04.253991 307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:04.254185 307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [term 1 LEADER]: Becoming Leader. State: Replica: 1ab7f53ff5194e848de277ee27020cc8, State: Running, Role: LEADER
I20240502 14:00:04.254513 307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab7f53ff5194e848de277ee27020cc8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42179 } }
I20240502 14:00:04.254619 302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 14:00:04.255923 308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ab7f53ff5194e848de277ee27020cc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab7f53ff5194e848de277ee27020cc8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42179 } } }
I20240502 14:00:04.256058 308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [sys.catalog]: This master's current role is: LEADER
I20240502 14:00:04.256398 316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 14:00:04.256340 309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ab7f53ff5194e848de277ee27020cc8. Latest consensus state: current_term: 1 leader_uuid: "1ab7f53ff5194e848de277ee27020cc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab7f53ff5194e848de277ee27020cc8" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 42179 } } }
I20240502 14:00:04.256534 309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8 [sys.catalog]: This master's current role is: LEADER
I20240502 14:00:04.259042 316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 14:00:04.264194 316 catalog_manager.cc:1347] Generated new cluster ID: 1bb40bc272c946a183f4db8f8193183c
I20240502 14:00:04.264257 316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 14:00:04.281836 316 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 14:00:04.283001 316 catalog_manager.cc:1504] Loading token signing keys...
I20240502 14:00:04.292896 316 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Generated new TSK 0
I20240502 14:00:04.293709 316 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 14:00:04.303092 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.129:0
--local_ip_for_outbound_sockets=127.26.25.129
--webserver_interface=127.26.25.129
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:00:04.409813 326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:04.410054 326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:04.410126 326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:04.413398 326 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:04.413471 326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:04.413563 326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.129
I20240502 14:00:04.417387 326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.26.25.129
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.129
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:04.418255 326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:04.419329 326 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:04.427531 332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:04.428037 331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:04.428647 334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:04.429605 326 server_base.cc:1034] running on GCE node
I20240502 14:00:04.429921 326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:04.430415 326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:04.431571 326 hybrid_clock.cc:648] HybridClock initialized: now 1714658404431546 us; error 37 us; skew 500 ppm
I20240502 14:00:04.433427 326 webserver.cc:460] Webserver started at http://127.26.25.129:44469/ using document root <none> and password file <none>
I20240502 14:00:04.433984 326 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:04.434039 326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:04.434216 326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:04.435885 326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "f003b732dd0f46958262b42f8d20305e"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.436359 326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "f003b732dd0f46958262b42f8d20305e"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.439488 326 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20240502 14:00:04.441517 340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:04.442463 326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:04.442577 326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "f003b732dd0f46958262b42f8d20305e"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.442657 326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:04.453938 326 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:04.454493 326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:04.454648 326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:04.455184 326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:04.456044 326 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:04.456094 326 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:04.456148 326 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:04.456183 326 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:04.464695 326 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.129:44715
I20240502 14:00:04.464751 455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.129:44715 every 8 connection(s)
I20240502 14:00:04.465611 326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 14:00:04.468802 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 326
I20240502 14:00:04.468936 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 14:00:04.471103 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.130:0
--local_ip_for_outbound_sockets=127.26.25.130
--webserver_interface=127.26.25.130
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:04.474743 456 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:04.474956 456 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:04.475433 456 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:04.477037 32734 ts_manager.cc:194] Registered new tserver with Master: f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:04.478811 32734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.129:45911
W20240502 14:00:04.577018 459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:04.577234 459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:04.577275 459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:04.580200 459 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:04.580255 459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:04.580317 459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.130
I20240502 14:00:04.583786 459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.26.25.130
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.130
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:04.584615 459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:04.585662 459 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:04.594269 464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:04.594852 467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:04.595351 465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:04.596247 459 server_base.cc:1034] running on GCE node
I20240502 14:00:04.596628 459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:04.597179 459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:04.598327 459 hybrid_clock.cc:648] HybridClock initialized: now 1714658404598301 us; error 41 us; skew 500 ppm
I20240502 14:00:04.600195 459 webserver.cc:460] Webserver started at http://127.26.25.130:38017/ using document root <none> and password file <none>
I20240502 14:00:04.600730 459 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:04.600783 459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:04.600961 459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:04.602525 459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "c351a89a5fe34e788b9d8064fbffa751"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.602991 459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "c351a89a5fe34e788b9d8064fbffa751"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.606202 459 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240502 14:00:04.608356 473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:04.609388 459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:00:04.609519 459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "c351a89a5fe34e788b9d8064fbffa751"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.609607 459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:04.641005 459 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:04.641762 459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:04.641925 459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:04.642431 459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:04.643252 459 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:04.643301 459 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:04.643357 459 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:04.643388 459 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:04.651769 459 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.130:35251
I20240502 14:00:04.651796 587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.130:35251 every 8 connection(s)
I20240502 14:00:04.652693 459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 14:00:04.657232 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 459
I20240502 14:00:04.657363 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 14:00:04.659704 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.131:0
--local_ip_for_outbound_sockets=127.26.25.131
--webserver_interface=127.26.25.131
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:04.661767 588 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:04.661962 588 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:04.662428 588 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:04.663414 32734 ts_manager.cc:194] Registered new tserver with Master: c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:04.664237 32734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.130:41275
W20240502 14:00:04.764166 591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:04.764428 591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:04.764475 591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:04.767376 591 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:04.767436 591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:04.767510 591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.131
I20240502 14:00:04.770964 591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.26.25.131
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.131
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:04.771759 591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:04.772862 591 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:04.781261 596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:04.782011 599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:04.782783 597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:04.783418 591 server_base.cc:1034] running on GCE node
I20240502 14:00:04.783798 591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:04.784343 591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:04.785496 591 hybrid_clock.cc:648] HybridClock initialized: now 1714658404785478 us; error 34 us; skew 500 ppm
I20240502 14:00:04.793754 591 webserver.cc:460] Webserver started at http://127.26.25.131:42863/ using document root <none> and password file <none>
I20240502 14:00:04.794309 591 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:04.794395 591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:04.794595 591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:04.796269 591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "58107b611f8f475a910a6fd54dfd1fb9"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.796731 591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "58107b611f8f475a910a6fd54dfd1fb9"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.799961 591 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:00:04.802119 605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:04.803035 591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:04.803140 591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "58107b611f8f475a910a6fd54dfd1fb9"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.803220 591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:04.821671 591 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:04.822235 591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:04.822392 591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:04.822896 591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:04.823753 591 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:04.823805 591 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:04.823913 591 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:04.823951 591 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:04.832090 591 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.131:38887
I20240502 14:00:04.832119 719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.131:38887 every 8 connection(s)
I20240502 14:00:04.832969 591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 14:00:04.835337 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 591
I20240502 14:00:04.835446 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 14:00:04.837590 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.132:0
--local_ip_for_outbound_sockets=127.26.25.132
--webserver_interface=127.26.25.132
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:04.841822 720 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:04.842029 720 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:04.842525 720 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:04.843473 32734 ts_manager.cc:194] Registered new tserver with Master: 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:04.844043 32734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.131:45267
W20240502 14:00:04.941851 723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:04.942075 723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:04.942145 723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:04.945117 723 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:04.945189 723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:04.945279 723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.132
I20240502 14:00:04.948848 723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.26.25.132
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.132
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:04.949713 723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:04.950793 723 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:04.959395 728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:04.959911 729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:04.960791 731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:04.961491 723 server_base.cc:1034] running on GCE node
I20240502 14:00:04.961863 723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:04.962421 723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:04.963575 723 hybrid_clock.cc:648] HybridClock initialized: now 1714658404963550 us; error 36 us; skew 500 ppm
I20240502 14:00:04.965620 723 webserver.cc:460] Webserver started at http://127.26.25.132:37231/ using document root <none> and password file <none>
I20240502 14:00:04.966150 723 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:04.966207 723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:04.966392 723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:04.968086 723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "478dfa53d7944f38b4e6cabee21b2d01"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.968531 723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "478dfa53d7944f38b4e6cabee21b2d01"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.971629 723 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240502 14:00:04.973644 737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:04.974613 723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:04.974725 723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "478dfa53d7944f38b4e6cabee21b2d01"
format_stamp: "Formatted at 2024-05-02 14:00:04 on dist-test-slave-sldb"
I20240502 14:00:04.974819 723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:05.007104 723 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:05.007812 723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:05.008024 723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:05.008667 723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:05.009729 723 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:05.009778 723 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.009846 723 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:05.009873 723 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.018227 723 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.132:33443
I20240502 14:00:05.018249 851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.132:33443 every 8 connection(s)
I20240502 14:00:05.019124 723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 14:00:05.023734 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 723
I20240502 14:00:05.023888 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 14:00:05.025991 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.133:0
--local_ip_for_outbound_sockets=127.26.25.133
--webserver_interface=127.26.25.133
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:05.027925 852 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:05.028155 852 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:05.028756 852 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:05.029760 32734 ts_manager.cc:194] Registered new tserver with Master: 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:05.030277 32734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.132:42677
W20240502 14:00:05.130697 855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:05.130893 855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:05.130934 855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:05.133852 855 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:05.133909 855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:05.133971 855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.133
I20240502 14:00:05.137405 855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.25.133
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.133
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:05.138166 855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:05.139185 855 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:05.147245 860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:05.147792 863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:05.148411 861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:05.149339 855 server_base.cc:1034] running on GCE node
I20240502 14:00:05.149664 855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:05.150158 855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:05.151297 855 hybrid_clock.cc:648] HybridClock initialized: now 1714658405151286 us; error 24 us; skew 500 ppm
I20240502 14:00:05.153002 855 webserver.cc:460] Webserver started at http://127.26.25.133:33117/ using document root <none> and password file <none>
I20240502 14:00:05.153508 855 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:05.153587 855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:05.153770 855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:05.155364 855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "4bcd7712aa844d98977a7d7e605c31f5"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.155870 855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "4bcd7712aa844d98977a7d7e605c31f5"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.158987 855 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:05.161005 869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.161940 855 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:05.162039 855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "4bcd7712aa844d98977a7d7e605c31f5"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.162130 855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:05.189651 855 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:05.190250 855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:05.190407 855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:05.190918 855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:05.191848 855 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:05.191913 855 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.191984 855 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:05.192015 855 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.200038 855 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.133:34751
I20240502 14:00:05.200081 982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.133:34751 every 8 connection(s)
I20240502 14:00:05.200908 855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 14:00:05.201814 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 855
I20240502 14:00:05.201941 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 14:00:05.204216 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.134:0
--local_ip_for_outbound_sockets=127.26.25.134
--webserver_interface=127.26.25.134
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:05.210255 983 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:05.210515 983 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:05.211175 983 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:05.212257 32734 ts_manager.cc:194] Registered new tserver with Master: 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:05.212821 32734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.133:50375
W20240502 14:00:05.312129 986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:05.312337 986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:05.312391 986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:05.315505 986 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:05.315570 986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:05.315665 986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.134
I20240502 14:00:05.319242 986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.26.25.134
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.134
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:05.320184 986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:05.321259 986 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:05.330170 992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:05.330798 994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:05.331441 991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:05.332238 986 server_base.cc:1034] running on GCE node
I20240502 14:00:05.332629 986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:05.333168 986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:05.334316 986 hybrid_clock.cc:648] HybridClock initialized: now 1714658405334301 us; error 29 us; skew 500 ppm
I20240502 14:00:05.336273 986 webserver.cc:460] Webserver started at http://127.26.25.134:40103/ using document root <none> and password file <none>
I20240502 14:00:05.336817 986 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:05.336874 986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:05.337064 986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:05.340919 986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "452ecbc085e54db49c3b01c20a96f860"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.341405 986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "452ecbc085e54db49c3b01c20a96f860"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.344744 986 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:00:05.346868 1000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.347934 986 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:00:05.348047 986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "452ecbc085e54db49c3b01c20a96f860"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.348129 986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:05.365065 986 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:05.365662 986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:05.365809 986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:05.366333 986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:05.367161 986 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:05.367208 986 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.367269 986 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:05.367295 986 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.375548 986 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.134:39061
I20240502 14:00:05.375583 1114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.134:39061 every 8 connection(s)
I20240502 14:00:05.376467 986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 14:00:05.380599 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 986
I20240502 14:00:05.380724 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 14:00:05.383092 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.135:0
--local_ip_for_outbound_sockets=127.26.25.135
--webserver_interface=127.26.25.135
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:05.385555 1115 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:05.385757 1115 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:05.386294 1115 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:05.387332 32734 ts_manager.cc:194] Registered new tserver with Master: 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:05.388177 32734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.134:59513
I20240502 14:00:05.481010 456 heartbeater.cc:503] Master 127.26.25.190:42179 was elected leader, sending a full tablet report...
W20240502 14:00:05.488687 1118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:05.488873 1118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:05.488912 1118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:05.491914 1118 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:05.491971 1118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:05.492034 1118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.135
I20240502 14:00:05.495456 1118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.26.25.135
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.1118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.135
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:05.496315 1118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:05.497398 1118 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:05.505648 1124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:05.506225 1126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:05.507107 1123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:05.507877 1118 server_base.cc:1034] running on GCE node
I20240502 14:00:05.508198 1118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:05.508685 1118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:05.509830 1118 hybrid_clock.cc:648] HybridClock initialized: now 1714658405509821 us; error 23 us; skew 500 ppm
I20240502 14:00:05.511564 1118 webserver.cc:460] Webserver started at http://127.26.25.135:37277/ using document root <none> and password file <none>
I20240502 14:00:05.512115 1118 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:05.512171 1118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:05.512328 1118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:05.514047 1118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "2b6796cf0e2e496e9dd641beb432f485"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.514470 1118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "2b6796cf0e2e496e9dd641beb432f485"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.517694 1118 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20240502 14:00:05.519680 1132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.520664 1118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:05.520753 1118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "2b6796cf0e2e496e9dd641beb432f485"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.520823 1118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:05.544750 1118 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:05.545256 1118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:05.545379 1118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:05.545914 1118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:05.546721 1118 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:05.546778 1118 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.546844 1118 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:05.546870 1118 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.555141 1118 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.135:45509
I20240502 14:00:05.555169 1245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.135:45509 every 8 connection(s)
I20240502 14:00:05.556200 1118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 14:00:05.559083 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 1118
I20240502 14:00:05.559190 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 14:00:05.561539 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.136:0
--local_ip_for_outbound_sockets=127.26.25.136
--webserver_interface=127.26.25.136
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:05.565570 1246 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:05.565814 1246 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:05.566471 1246 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:05.567502 32734 ts_manager.cc:194] Registered new tserver with Master: 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:05.568056 32734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.135:53525
I20240502 14:00:05.666085 588 heartbeater.cc:503] Master 127.26.25.190:42179 was elected leader, sending a full tablet report...
W20240502 14:00:05.667053 1250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:05.667260 1250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:05.667320 1250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:05.670377 1250 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:05.670441 1250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:05.670514 1250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.136
I20240502 14:00:05.674050 1250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.26.25.136
--webserver_port=0
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.1250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.136
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:05.674921 1250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:05.676010 1250 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:05.684756 1256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:05.685197 1255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:05.686046 1258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:05.687109 1250 server_base.cc:1034] running on GCE node
I20240502 14:00:05.687528 1250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:05.688138 1250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:05.689285 1250 hybrid_clock.cc:648] HybridClock initialized: now 1714658405689275 us; error 31 us; skew 500 ppm
I20240502 14:00:05.691131 1250 webserver.cc:460] Webserver started at http://127.26.25.136:38065/ using document root <none> and password file <none>
I20240502 14:00:05.691665 1250 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:05.691726 1250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:05.691948 1250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:05.693576 1250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "da3f8160557e454f8547f45e80c2f013"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.694049 1250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "da3f8160557e454f8547f45e80c2f013"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.697299 1250 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20240502 14:00:05.699393 1264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.700376 1250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:05.700481 1250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "da3f8160557e454f8547f45e80c2f013"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:05.700568 1250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:05.721443 1250 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:05.722031 1250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:05.722189 1250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:05.722702 1250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:05.723585 1250 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:05.723634 1250 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.723701 1250 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:05.723727 1250 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:05.731776 1250 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.136:43131
I20240502 14:00:05.731851 1377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.136:43131 every 8 connection(s)
I20240502 14:00:05.732721 1250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 14:00:05.737717 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 1250
I20240502 14:00:05.737834 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 14:00:05.741309 1378 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:05.741521 1378 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:05.742007 1378 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:05.742873 32734 ts_manager.cc:194] Registered new tserver with Master: da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:05.743398 32734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.136:53363
I20240502 14:00:05.751781 26726 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 14:00:05.760911 26726 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 14:00:05.760995 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 855
I20240502 14:00:05.766667 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 986
I20240502 14:00:05.771785 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 1118
I20240502 14:00:05.777302 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 1250
I20240502 14:00:05.845782 720 heartbeater.cc:503] Master 127.26.25.190:42179 was elected leader, sending a full tablet report...
I20240502 14:00:06.032686 852 heartbeater.cc:503] Master 127.26.25.190:42179 was elected leader, sending a full tablet report...
I20240502 14:00:09.534333 32734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:09.554212 388 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac54cbde3184904a66f2383cbc3ca8a (DEFAULT_TABLE table=rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:09.554198 786 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac54cbde3184904a66f2383cbc3ca8a (DEFAULT_TABLE table=rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:09.555223 786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ac54cbde3184904a66f2383cbc3ca8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:09.555223 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ac54cbde3184904a66f2383cbc3ca8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:09.556387 521 tablet_service.cc:1467] Processing CreateTablet for tablet 50fbf855074e4c5c89064df2bbf59c06 (DEFAULT_TABLE table=rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:09.557268 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50fbf855074e4c5c89064df2bbf59c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:09.557166 654 tablet_service.cc:1467] Processing CreateTablet for tablet 50fbf855074e4c5c89064df2bbf59c06 (DEFAULT_TABLE table=rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:09.557945 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50fbf855074e4c5c89064df2bbf59c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:09.554212 386 tablet_service.cc:1467] Processing CreateTablet for tablet 0b5c9e82af1e4db192609694441e3705 (DEFAULT_TABLE table=rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:09.560148 653 tablet_service.cc:1467] Processing CreateTablet for tablet 0b5c9e82af1e4db192609694441e3705 (DEFAULT_TABLE table=rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:09.560397 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b5c9e82af1e4db192609694441e3705. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:09.560400 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b5c9e82af1e4db192609694441e3705. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:09.561115 785 tablet_service.cc:1467] Processing CreateTablet for tablet 0b5c9e82af1e4db192609694441e3705 (DEFAULT_TABLE table=rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:09.561358 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b5c9e82af1e4db192609694441e3705. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:09.562722 1398 tablet_bootstrap.cc:492] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:09.563659 522 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac54cbde3184904a66f2383cbc3ca8a (DEFAULT_TABLE table=rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:09.563956 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ac54cbde3184904a66f2383cbc3ca8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:09.554210 387 tablet_service.cc:1467] Processing CreateTablet for tablet 50fbf855074e4c5c89064df2bbf59c06 (DEFAULT_TABLE table=rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:09.564507 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50fbf855074e4c5c89064df2bbf59c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:09.565634 1398 tablet_bootstrap.cc:654] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:09.566819 1398 log.cc:824] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:09.567557 1400 tablet_bootstrap.cc:492] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:09.567725 1399 tablet_bootstrap.cc:492] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:09.568543 1402 tablet_bootstrap.cc:492] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:09.569974 1398 tablet_bootstrap.cc:492] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:09.570183 1399 tablet_bootstrap.cc:654] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:09.570216 1398 ts_tablet_manager.cc:1367] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20240502 14:00:09.570294 1400 tablet_bootstrap.cc:654] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:09.570509 1402 tablet_bootstrap.cc:654] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:09.571087 1400 log.cc:824] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:09.571087 1399 log.cc:824] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:09.571271 1402 log.cc:824] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:09.572811 1399 tablet_bootstrap.cc:492] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:09.572964 1402 tablet_bootstrap.cc:492] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:09.573096 1399 ts_tablet_manager.cc:1367] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20240502 14:00:09.573130 1402 ts_tablet_manager.cc:1367] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 14:00:09.573657 1398 raft_consensus.cc:348] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.573984 1400 tablet_bootstrap.cc:492] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:09.573980 1398 raft_consensus.cc:374] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:09.574043 1398 raft_consensus.cc:729] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:09.574209 1400 ts_tablet_manager.cc:1367] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20240502 14:00:09.574604 1398 consensus_queue.cc:260] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.575664 1398 ts_tablet_manager.cc:1398] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240502 14:00:09.576757 1402 raft_consensus.cc:348] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:09.576969 1402 raft_consensus.cc:374] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:09.577032 1402 raft_consensus.cc:729] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:09.577167 1398 tablet_bootstrap.cc:492] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:09.577440 1402 consensus_queue.cc:260] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:09.577997 1400 raft_consensus.cc:348] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:09.578300 1398 tablet_bootstrap.cc:654] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:09.578356 1402 ts_tablet_manager.cc:1398] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20240502 14:00:09.578967 1402 tablet_bootstrap.cc:492] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:09.579455 1399 raft_consensus.cc:348] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.579713 1399 raft_consensus.cc:374] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:09.579859 1399 raft_consensus.cc:729] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:09.580297 1399 consensus_queue.cc:260] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.580718 1402 tablet_bootstrap.cc:654] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:09.580973 1400 raft_consensus.cc:374] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:09.581225 1399 ts_tablet_manager.cc:1398] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.008s user 0.003s sys 0.002s
I20240502 14:00:09.581223 1400 raft_consensus.cc:729] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:09.581682 1400 consensus_queue.cc:260] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:09.582151 1398 tablet_bootstrap.cc:492] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:09.582237 1398 ts_tablet_manager.cc:1367] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:09.582584 1399 tablet_bootstrap.cc:492] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:09.582655 1400 ts_tablet_manager.cc:1398] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.008s user 0.000s sys 0.005s
I20240502 14:00:09.582752 1398 raft_consensus.cc:348] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:09.582854 1398 raft_consensus.cc:374] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:09.582913 1398 raft_consensus.cc:729] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:09.582988 1400 tablet_bootstrap.cc:492] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:09.583026 1398 consensus_queue.cc:260] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:09.583284 1398 ts_tablet_manager.cc:1398] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:09.583683 1399 tablet_bootstrap.cc:654] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:09.584594 1400 tablet_bootstrap.cc:654] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:09.584847 1402 tablet_bootstrap.cc:492] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:09.584939 1402 ts_tablet_manager.cc:1367] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 14:00:09.585490 1402 raft_consensus.cc:348] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.585602 1402 raft_consensus.cc:374] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:09.585641 1402 raft_consensus.cc:729] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:09.585778 1402 consensus_queue.cc:260] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.586076 1402 ts_tablet_manager.cc:1398] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:09.586109 1399 tablet_bootstrap.cc:492] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:09.586189 1399 ts_tablet_manager.cc:1367] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 14:00:09.586187 1400 tablet_bootstrap.cc:492] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:09.586266 1400 ts_tablet_manager.cc:1367] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:09.586704 1399 raft_consensus.cc:348] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.586795 1399 raft_consensus.cc:374] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:09.586827 1399 raft_consensus.cc:729] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:09.586800 1400 raft_consensus.cc:348] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.586906 1400 raft_consensus.cc:374] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:09.586959 1400 raft_consensus.cc:729] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:09.586947 1399 consensus_queue.cc:260] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.587071 1400 consensus_queue.cc:260] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.587266 1399 ts_tablet_manager.cc:1398] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:09.587347 1400 ts_tablet_manager.cc:1398] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:09.587510 1400 tablet_bootstrap.cc:492] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
W20240502 14:00:09.587987 721 tablet.cc:2369] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:09.588335 1400 tablet_bootstrap.cc:654] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:09.589385 1400 tablet_bootstrap.cc:492] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:09.589458 1400 ts_tablet_manager.cc:1367] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:09.589908 1400 raft_consensus.cc:348] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.589995 1400 raft_consensus.cc:374] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:09.590016 1400 raft_consensus.cc:729] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:09.590145 1400 consensus_queue.cc:260] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.590380 1400 ts_tablet_manager.cc:1398] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20240502 14:00:09.657027 589 tablet.cc:2369] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:09.682108 1411 raft_consensus.cc:482] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:09.682314 1411 raft_consensus.cc:504] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.683378 1411 leader_election.cc:288] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:09.685683 1409 raft_consensus.cc:482] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:09.685851 1409 raft_consensus.cc:504] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.686959 1409 leader_election.cc:288] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:09.687726 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50fbf855074e4c5c89064df2bbf59c06" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:09.688053 674 raft_consensus.cc:2454] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:09.688570 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50fbf855074e4c5c89064df2bbf59c06" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:09.688731 542 raft_consensus.cc:2454] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:09.688817 341 leader_election.cc:302] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:09.689389 1411 raft_consensus.cc:2781] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:09.689469 1411 raft_consensus.cc:482] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:09.689545 1411 raft_consensus.cc:3037] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:09.690770 1411 raft_consensus.cc:504] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.691169 1411 leader_election.cc:288] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:09.691527 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50fbf855074e4c5c89064df2bbf59c06" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:09.691637 542 raft_consensus.cc:3037] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:09.691792 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50fbf855074e4c5c89064df2bbf59c06" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:09.691808 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50fbf855074e4c5c89064df2bbf59c06" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:09.691923 674 raft_consensus.cc:3037] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:09.692035 409 raft_consensus.cc:2379] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c351a89a5fe34e788b9d8064fbffa751 in current term 1: Already voted for candidate f003b732dd0f46958262b42f8d20305e in this term.
I20240502 14:00:09.692765 674 raft_consensus.cc:2454] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:09.692996 542 raft_consensus.cc:2454] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:09.693071 341 leader_election.cc:302] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:09.693238 1411 raft_consensus.cc:2781] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:09.693434 1411 raft_consensus.cc:686] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:09.693481 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50fbf855074e4c5c89064df2bbf59c06" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:09.693634 674 raft_consensus.cc:2379] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c351a89a5fe34e788b9d8064fbffa751 in current term 1: Already voted for candidate f003b732dd0f46958262b42f8d20305e in this term.
I20240502 14:00:09.693760 1411 consensus_queue.cc:237] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.694005 474 leader_election.cc:302] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751; no voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e
I20240502 14:00:09.694221 1409 raft_consensus.cc:2726] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 14:00:09.696770 32732 catalog_manager.cc:5484] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:09.720551 457 tablet.cc:2369] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:09.735095 1409 raft_consensus.cc:482] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:09.735232 1409 raft_consensus.cc:504] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.735694 1409 leader_election.cc:288] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:09.736192 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac54cbde3184904a66f2383cbc3ca8a" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:09.736377 409 raft_consensus.cc:2454] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:09.736693 476 leader_election.cc:302] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:09.736853 1409 raft_consensus.cc:2781] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:09.736927 1409 raft_consensus.cc:482] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:09.736970 1409 raft_consensus.cc:3037] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:09.737967 1409 raft_consensus.cc:504] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.738286 1409 leader_election.cc:288] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:09.738660 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac54cbde3184904a66f2383cbc3ca8a" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:09.738782 409 raft_consensus.cc:3037] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:09.739440 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac54cbde3184904a66f2383cbc3ca8a" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:09.739449 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac54cbde3184904a66f2383cbc3ca8a" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:09.739737 806 raft_consensus.cc:2454] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:09.739789 409 raft_consensus.cc:2454] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:09.740110 476 leader_election.cc:302] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:09.740341 1409 raft_consensus.cc:2781] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:09.740551 1409 raft_consensus.cc:686] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:09.740837 1409 consensus_queue.cc:237] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:09.743785 32732 catalog_manager.cc:5484] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } }
W20240502 14:00:09.773300 853 tablet.cc:2369] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:09.912393 1406 raft_consensus.cc:482] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:09.912545 1406 raft_consensus.cc:504] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:09.913779 1406 leader_election.cc:288] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:09.918010 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b5c9e82af1e4db192609694441e3705" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:09.918183 674 raft_consensus.cc:2454] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:09.918421 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b5c9e82af1e4db192609694441e3705" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:09.918550 409 raft_consensus.cc:2454] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:09.918531 738 leader_election.cc:302] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:09.918776 1406 raft_consensus.cc:2781] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:09.918849 1406 raft_consensus.cc:482] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:09.918879 1406 raft_consensus.cc:3037] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:09.919755 1406 raft_consensus.cc:504] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:09.920137 1406 leader_election.cc:288] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:09.920466 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b5c9e82af1e4db192609694441e3705" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:09.920585 674 raft_consensus.cc:3037] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:09.920625 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b5c9e82af1e4db192609694441e3705" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:09.920749 409 raft_consensus.cc:3037] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:09.921350 674 raft_consensus.cc:2454] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:09.921497 409 raft_consensus.cc:2454] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:09.921631 738 leader_election.cc:302] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:09.921808 1406 raft_consensus.cc:2781] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:09.922019 1406 raft_consensus.cc:686] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:09.922353 1406 consensus_queue.cc:237] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:09.925376 32732 catalog_manager.cc:5484] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:10.114678 1427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:10.114956 1427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:10.118028 1427 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 14:00:10.131361 1411 consensus_queue.cc:1035] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:10.136947 1441 consensus_queue.cc:1035] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_0'
I20240502 14:00:10.217895 409 raft_consensus.cc:1261] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:10.218590 805 raft_consensus.cc:3037] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:10.219740 805 raft_consensus.cc:1261] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:10.221038 674 raft_consensus.cc:1261] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:10.221670 1409 consensus_queue.cc:1035] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:10.221700 1406 consensus_queue.cc:1035] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:10.222662 1409 consensus_queue.cc:1035] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:10.224920 409 raft_consensus.cc:1261] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:10.225852 1425 consensus_queue.cc:1035] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:10.238068 1478 mvcc.cc:205] Tried to move back new op lower bound from 7023240848271372288 to 7023240847044452352. Current Snapshot: MvccSnapshot[applied={T|T < 7023240848248283136}]
I20240502 14:00:10.238186 1472 mvcc.cc:205] Tried to move back new op lower bound from 7023240848271372288 to 7023240847044452352. Current Snapshot: MvccSnapshot[applied={T|T < 7023240848248283136}]
I20240502 14:00:10.257591 1444 mvcc.cc:205] Tried to move back new op lower bound from 7023240848265711616 to 7023240846300823552. Current Snapshot: MvccSnapshot[applied={T|T < 7023240848246509568}]
I20240502 14:00:10.259130 1442 mvcc.cc:205] Tried to move back new op lower bound from 7023240848248283136 to 7023240847044452352. Current Snapshot: MvccSnapshot[applied={T|T < 7023240848248283136}]
INSERT report
rows total: 2000
time total: 129.671 ms
time per row: 0.0648357 ms
I20240502 14:00:10.293684 32734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:10.299018 522 tablet_service.cc:1467] Processing CreateTablet for tablet 76f41f7f19c94930aaba452854be2aaa (DEFAULT_TABLE table=rebalance_test_table_1 [id=2b9baba891764826815587748cbcfcf1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:10.299234 654 tablet_service.cc:1467] Processing CreateTablet for tablet 76f41f7f19c94930aaba452854be2aaa (DEFAULT_TABLE table=rebalance_test_table_1 [id=2b9baba891764826815587748cbcfcf1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:10.299315 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f41f7f19c94930aaba452854be2aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:10.299535 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f41f7f19c94930aaba452854be2aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:10.300006 520 tablet_service.cc:1467] Processing CreateTablet for tablet e673d6e79c0f4c62a393a542473930b1 (DEFAULT_TABLE table=rebalance_test_table_1 [id=2b9baba891764826815587748cbcfcf1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:10.300135 785 tablet_service.cc:1467] Processing CreateTablet for tablet 76f41f7f19c94930aaba452854be2aaa (DEFAULT_TABLE table=rebalance_test_table_1 [id=2b9baba891764826815587748cbcfcf1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:10.300252 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e673d6e79c0f4c62a393a542473930b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:10.300376 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f41f7f19c94930aaba452854be2aaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:10.300429 388 tablet_service.cc:1467] Processing CreateTablet for tablet e673d6e79c0f4c62a393a542473930b1 (DEFAULT_TABLE table=rebalance_test_table_1 [id=2b9baba891764826815587748cbcfcf1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:10.300691 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e673d6e79c0f4c62a393a542473930b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:10.301954 521 tablet_service.cc:1467] Processing CreateTablet for tablet a2c3caa06e284ef2ad50c6badfa8f59e (DEFAULT_TABLE table=rebalance_test_table_1 [id=2b9baba891764826815587748cbcfcf1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:10.302193 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2c3caa06e284ef2ad50c6badfa8f59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:10.303007 653 tablet_service.cc:1467] Processing CreateTablet for tablet e673d6e79c0f4c62a393a542473930b1 (DEFAULT_TABLE table=rebalance_test_table_1 [id=2b9baba891764826815587748cbcfcf1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:10.303287 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e673d6e79c0f4c62a393a542473930b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:10.303421 1495 tablet_bootstrap.cc:492] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:10.304447 654 tablet_service.cc:1467] Processing CreateTablet for tablet a2c3caa06e284ef2ad50c6badfa8f59e (DEFAULT_TABLE table=rebalance_test_table_1 [id=2b9baba891764826815587748cbcfcf1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:10.304611 1495 tablet_bootstrap.cc:654] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:10.304659 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2c3caa06e284ef2ad50c6badfa8f59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:10.305639 785 tablet_service.cc:1467] Processing CreateTablet for tablet a2c3caa06e284ef2ad50c6badfa8f59e (DEFAULT_TABLE table=rebalance_test_table_1 [id=2b9baba891764826815587748cbcfcf1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:10.305810 1493 tablet_bootstrap.cc:492] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:10.305927 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2c3caa06e284ef2ad50c6badfa8f59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:10.306701 1495 tablet_bootstrap.cc:492] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:10.306761 1493 tablet_bootstrap.cc:654] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:10.306782 1495 ts_tablet_manager.cc:1367] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:10.307257 1495 raft_consensus.cc:348] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.307358 1495 raft_consensus.cc:374] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:10.307389 1495 raft_consensus.cc:729] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:10.307554 1495 consensus_queue.cc:260] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.307793 1493 tablet_bootstrap.cc:492] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:10.307849 1495 ts_tablet_manager.cc:1398] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:10.307898 1493 ts_tablet_manager.cc:1367] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:10.308012 1495 tablet_bootstrap.cc:492] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:10.308348 1496 tablet_bootstrap.cc:492] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:10.308317 1493 raft_consensus.cc:348] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.308409 1493 raft_consensus.cc:374] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:10.308444 1493 raft_consensus.cc:729] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:10.308598 1493 consensus_queue.cc:260] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.308836 1495 tablet_bootstrap.cc:654] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:10.308928 1493 ts_tablet_manager.cc:1398] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:10.309069 1493 tablet_bootstrap.cc:492] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:10.309967 1493 tablet_bootstrap.cc:654] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:10.310178 1496 tablet_bootstrap.cc:654] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:10.310907 1494 tablet_bootstrap.cc:492] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:10.311056 1493 tablet_bootstrap.cc:492] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:10.311125 1493 ts_tablet_manager.cc:1367] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:10.311530 1495 tablet_bootstrap.cc:492] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:10.311607 1495 ts_tablet_manager.cc:1367] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:10.311556 1493 raft_consensus.cc:348] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.311645 1493 raft_consensus.cc:374] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:10.311681 1493 raft_consensus.cc:729] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:10.311843 1493 consensus_queue.cc:260] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.311951 1494 tablet_bootstrap.cc:654] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:10.312100 1493 ts_tablet_manager.cc:1398] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:10.312067 1495 raft_consensus.cc:348] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:10.312177 1495 raft_consensus.cc:374] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:10.312213 1495 raft_consensus.cc:729] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:10.312249 1493 tablet_bootstrap.cc:492] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:10.312330 1495 consensus_queue.cc:260] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:10.312624 1495 ts_tablet_manager.cc:1398] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:10.313040 1493 tablet_bootstrap.cc:654] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:10.313051 1494 tablet_bootstrap.cc:492] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:10.313133 1494 ts_tablet_manager.cc:1367] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:10.313632 1494 raft_consensus.cc:348] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.313741 1494 raft_consensus.cc:374] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:10.313782 1494 raft_consensus.cc:729] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:10.313899 1494 consensus_queue.cc:260] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.314174 1494 ts_tablet_manager.cc:1398] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:10.314347 1494 tablet_bootstrap.cc:492] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:10.314846 1493 tablet_bootstrap.cc:492] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:10.314929 1493 ts_tablet_manager.cc:1367] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:10.315265 1494 tablet_bootstrap.cc:654] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:10.315564 1496 tablet_bootstrap.cc:492] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:10.315591 1493 raft_consensus.cc:348] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:10.315681 1496 ts_tablet_manager.cc:1367] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:10.315688 1493 raft_consensus.cc:374] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:10.315729 1493 raft_consensus.cc:729] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:10.315879 1493 consensus_queue.cc:260] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:10.316099 1493 ts_tablet_manager.cc:1398] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:10.316164 1496 raft_consensus.cc:348] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.316269 1496 raft_consensus.cc:374] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:10.316303 1496 raft_consensus.cc:729] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:10.316421 1496 consensus_queue.cc:260] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.316684 1496 ts_tablet_manager.cc:1398] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:10.318053 1494 tablet_bootstrap.cc:492] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:10.318198 1494 ts_tablet_manager.cc:1367] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:10.318662 1494 raft_consensus.cc:348] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:10.318771 1494 raft_consensus.cc:374] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:10.318809 1494 raft_consensus.cc:729] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:10.318933 1494 consensus_queue.cc:260] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:10.319200 1494 ts_tablet_manager.cc:1398] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:10.319398 1494 tablet_bootstrap.cc:492] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:10.320297 1494 tablet_bootstrap.cc:654] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:10.321300 1494 tablet_bootstrap.cc:492] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:10.321368 1494 ts_tablet_manager.cc:1367] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:10.321789 1494 raft_consensus.cc:348] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.321879 1494 raft_consensus.cc:374] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:10.321902 1494 raft_consensus.cc:729] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:10.322022 1494 consensus_queue.cc:260] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.322237 1494 ts_tablet_manager.cc:1398] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:10.341051 1438 raft_consensus.cc:482] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:10.341198 1438 raft_consensus.cc:504] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.342192 1438 leader_election.cc:288] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:10.346038 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e673d6e79c0f4c62a393a542473930b1" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:10.346086 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e673d6e79c0f4c62a393a542473930b1" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:10.346218 542 raft_consensus.cc:2454] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:10.346218 408 raft_consensus.cc:2454] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:10.346560 608 leader_election.cc:302] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:10.346769 1438 raft_consensus.cc:2781] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:10.346827 1438 raft_consensus.cc:482] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:10.346880 1438 raft_consensus.cc:3037] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:10.349272 1438 raft_consensus.cc:504] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.349648 1438 leader_election.cc:288] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:10.349896 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e673d6e79c0f4c62a393a542473930b1" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:10.349949 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e673d6e79c0f4c62a393a542473930b1" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:10.350001 542 raft_consensus.cc:3037] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:10.350078 408 raft_consensus.cc:3037] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:10.350723 542 raft_consensus.cc:2454] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:10.350848 408 raft_consensus.cc:2454] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:10.351033 606 leader_election.cc:302] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:10.351214 1438 raft_consensus.cc:2781] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:10.351408 1438 raft_consensus.cc:686] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:10.351688 1438 consensus_queue.cc:237] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.353915 32726 catalog_manager.cc:5484] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:10.506129 1425 raft_consensus.cc:482] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:10.506287 1425 raft_consensus.cc:504] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:10.506798 1425 leader_election.cc:288] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:10.507658 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c3caa06e284ef2ad50c6badfa8f59e" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:10.507843 674 raft_consensus.cc:2454] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:10.508129 738 leader_election.cc:302] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:10.508296 1425 raft_consensus.cc:2781] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:10.508353 1425 raft_consensus.cc:482] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:10.508381 1425 raft_consensus.cc:3037] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:10.509204 1425 raft_consensus.cc:504] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:10.509538 1425 leader_election.cc:288] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:10.509876 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c3caa06e284ef2ad50c6badfa8f59e" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:10.509996 674 raft_consensus.cc:3037] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:10.510413 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c3caa06e284ef2ad50c6badfa8f59e" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:10.510531 542 raft_consensus.cc:2454] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:10.510535 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c3caa06e284ef2ad50c6badfa8f59e" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:10.510687 541 raft_consensus.cc:3037] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:10.510843 674 raft_consensus.cc:2454] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:10.511161 738 leader_election.cc:302] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:10.511320 1425 raft_consensus.cc:2781] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:10.511410 1425 raft_consensus.cc:686] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:10.511564 1425 consensus_queue.cc:237] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:10.511620 541 raft_consensus.cc:2454] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:10.513307 32726 catalog_manager.cc:5484] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:10.701844 1482 raft_consensus.cc:482] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:10.702010 1482 raft_consensus.cc:504] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.702454 1482 leader_election.cc:288] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:10.702922 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f41f7f19c94930aaba452854be2aaa" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:10.702924 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f41f7f19c94930aaba452854be2aaa" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:10.703063 805 raft_consensus.cc:2454] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:10.703074 674 raft_consensus.cc:2454] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:10.703393 477 leader_election.cc:302] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:10.703585 1482 raft_consensus.cc:2781] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:10.703651 1482 raft_consensus.cc:482] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:10.703678 1482 raft_consensus.cc:3037] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:10.704478 1482 raft_consensus.cc:504] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.704775 1482 leader_election.cc:288] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:10.705096 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f41f7f19c94930aaba452854be2aaa" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:10.705123 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f41f7f19c94930aaba452854be2aaa" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:10.705226 805 raft_consensus.cc:3037] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:10.705225 674 raft_consensus.cc:3037] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:10.705987 805 raft_consensus.cc:2454] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:10.705998 674 raft_consensus.cc:2454] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:10.706300 474 leader_election.cc:302] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:10.706462 1482 raft_consensus.cc:2781] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:10.706568 1482 raft_consensus.cc:686] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:10.706748 1482 consensus_queue.cc:237] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:10.708536 32726 catalog_manager.cc:5484] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:10.834026 1438 consensus_queue.cc:1035] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:10.838173 1438 consensus_queue.cc:1035] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 14:00:10.865865 1514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:10.866103 1514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:10.869040 1514 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 14:00:10.941609 674 raft_consensus.cc:1261] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:10.941903 673 raft_consensus.cc:1261] T 76f41f7f19c94930aaba452854be2aaa P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:10.942416 1482 consensus_queue.cc:1035] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:10.942571 541 raft_consensus.cc:1261] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:10.943333 1406 consensus_queue.cc:1035] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:10.943599 1406 consensus_queue.cc:1035] T a2c3caa06e284ef2ad50c6badfa8f59e P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:10.946260 805 raft_consensus.cc:1261] T 76f41f7f19c94930aaba452854be2aaa P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:10.946820 1482 consensus_queue.cc:1035] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 101.74 ms
time per row: 0.05087 ms
I20240502 14:00:11.010164 32734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:11.015652 785 tablet_service.cc:1467] Processing CreateTablet for tablet ea8c5159a1e24471ab878ffb50a6df05 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.016023 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8c5159a1e24471ab878ffb50a6df05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.016109 786 tablet_service.cc:1467] Processing CreateTablet for tablet aa30d86ef587427db4213f5c178a744a (DEFAULT_TABLE table=rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.016307 786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa30d86ef587427db4213f5c178a744a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.016549 784 tablet_service.cc:1467] Processing CreateTablet for tablet 2432b3ab36a54fbfa108031f67634e2c (DEFAULT_TABLE table=rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.016758 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2432b3ab36a54fbfa108031f67634e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.017665 388 tablet_service.cc:1467] Processing CreateTablet for tablet ea8c5159a1e24471ab878ffb50a6df05 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.017941 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8c5159a1e24471ab878ffb50a6df05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.019963 1577 tablet_bootstrap.cc:492] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:11.020061 387 tablet_service.cc:1467] Processing CreateTablet for tablet aa30d86ef587427db4213f5c178a744a (DEFAULT_TABLE table=rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.020264 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa30d86ef587427db4213f5c178a744a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.020886 1577 tablet_bootstrap.cc:654] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.021287 654 tablet_service.cc:1467] Processing CreateTablet for tablet aa30d86ef587427db4213f5c178a744a (DEFAULT_TABLE table=rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.021610 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa30d86ef587427db4213f5c178a744a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.022007 1577 tablet_bootstrap.cc:492] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:11.022102 1577 ts_tablet_manager.cc:1367] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:11.022176 522 tablet_service.cc:1467] Processing CreateTablet for tablet ea8c5159a1e24471ab878ffb50a6df05 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.022426 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8c5159a1e24471ab878ffb50a6df05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.022532 1577 raft_consensus.cc:348] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.022640 1577 raft_consensus.cc:374] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.022682 1577 raft_consensus.cc:729] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.022827 1577 consensus_queue.cc:260] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.023128 1577 ts_tablet_manager.cc:1398] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.023308 1577 tablet_bootstrap.cc:492] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:11.024370 1577 tablet_bootstrap.cc:654] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.025146 521 tablet_service.cc:1467] Processing CreateTablet for tablet 2432b3ab36a54fbfa108031f67634e2c (DEFAULT_TABLE table=rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.025414 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2432b3ab36a54fbfa108031f67634e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.026899 1579 tablet_bootstrap.cc:492] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:11.028079 1577 tablet_bootstrap.cc:492] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:11.028172 1577 ts_tablet_manager.cc:1367] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:11.028620 1579 tablet_bootstrap.cc:654] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.028661 1577 raft_consensus.cc:348] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:11.028757 1577 raft_consensus.cc:374] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.028796 1577 raft_consensus.cc:729] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.028916 1577 consensus_queue.cc:260] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:11.029363 1577 ts_tablet_manager.cc:1398] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.029700 1576 tablet_bootstrap.cc:492] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:11.029815 1579 tablet_bootstrap.cc:492] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:11.029911 1579 ts_tablet_manager.cc:1367] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20240502 14:00:11.030351 1579 raft_consensus.cc:348] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:11.030454 1579 raft_consensus.cc:374] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.030505 1579 raft_consensus.cc:729] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.030594 1576 tablet_bootstrap.cc:654] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.030647 1579 consensus_queue.cc:260] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:11.030952 1579 ts_tablet_manager.cc:1398] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:11.031813 1576 tablet_bootstrap.cc:492] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:11.031934 1576 ts_tablet_manager.cc:1367] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:11.032389 1576 raft_consensus.cc:348] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:11.032497 1576 raft_consensus.cc:374] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.032536 1576 raft_consensus.cc:729] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.032527 388 tablet_service.cc:1467] Processing CreateTablet for tablet 2432b3ab36a54fbfa108031f67634e2c (DEFAULT_TABLE table=rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.032677 1576 consensus_queue.cc:260] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:11.032773 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2432b3ab36a54fbfa108031f67634e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.032948 1576 ts_tablet_manager.cc:1398] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:11.033098 1576 tablet_bootstrap.cc:492] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:11.034018 1576 tablet_bootstrap.cc:654] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.035751 1580 tablet_bootstrap.cc:492] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:11.035877 1577 tablet_bootstrap.cc:492] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:11.036484 1580 tablet_bootstrap.cc:654] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.036695 1577 tablet_bootstrap.cc:654] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.037542 1576 tablet_bootstrap.cc:492] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:11.037632 1576 ts_tablet_manager.cc:1367] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:11.037835 1577 tablet_bootstrap.cc:492] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:11.037911 1577 ts_tablet_manager.cc:1367] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:11.038028 1576 raft_consensus.cc:348] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.038113 1576 raft_consensus.cc:374] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.038154 1576 raft_consensus.cc:729] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.038269 1576 consensus_queue.cc:260] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.038286 1577 raft_consensus.cc:348] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.038388 1577 raft_consensus.cc:374] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.038414 1577 raft_consensus.cc:729] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.038516 1577 consensus_queue.cc:260] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.038542 1576 ts_tablet_manager.cc:1398] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.038690 1576 tablet_bootstrap.cc:492] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:11.038745 1577 ts_tablet_manager.cc:1398] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.039430 1576 tablet_bootstrap.cc:654] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.040390 1580 tablet_bootstrap.cc:492] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:11.040519 1580 ts_tablet_manager.cc:1367] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:11.040954 1580 raft_consensus.cc:348] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.041072 1576 tablet_bootstrap.cc:492] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:11.041115 1580 raft_consensus.cc:374] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.041158 1576 ts_tablet_manager.cc:1367] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:11.041162 1580 raft_consensus.cc:729] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.041299 1580 consensus_queue.cc:260] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.041538 1580 ts_tablet_manager.cc:1398] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.041682 1580 tablet_bootstrap.cc:492] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:11.041764 1576 raft_consensus.cc:348] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.041857 1576 raft_consensus.cc:374] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.041882 1576 raft_consensus.cc:729] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.041965 1576 consensus_queue.cc:260] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.042194 1576 ts_tablet_manager.cc:1398] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:11.042523 1580 tablet_bootstrap.cc:654] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.043661 1580 tablet_bootstrap.cc:492] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:11.043738 1580 ts_tablet_manager.cc:1367] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:11.044132 1580 raft_consensus.cc:348] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.044229 1580 raft_consensus.cc:374] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.044260 1580 raft_consensus.cc:729] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.044399 1580 consensus_queue.cc:260] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.044700 1580 ts_tablet_manager.cc:1398] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:11.063784 1411 raft_consensus.cc:482] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:11.063925 1411 raft_consensus.cc:504] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:11.064410 1411 leader_election.cc:288] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:11.064832 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa30d86ef587427db4213f5c178a744a" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:11.064992 674 raft_consensus.cc:2454] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:11.065275 341 leader_election.cc:302] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:11.065460 1411 raft_consensus.cc:2781] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:11.065534 1411 raft_consensus.cc:482] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:11.065570 1411 raft_consensus.cc:3037] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.066421 1411 raft_consensus.cc:504] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:11.066756 1411 leader_election.cc:288] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:11.067076 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa30d86ef587427db4213f5c178a744a" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:11.067189 674 raft_consensus.cc:3037] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.068060 674 raft_consensus.cc:2454] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:11.068125 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa30d86ef587427db4213f5c178a744a" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:11.068212 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa30d86ef587427db4213f5c178a744a" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:11.068243 805 raft_consensus.cc:2454] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:11.068349 341 leader_election.cc:302] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:11.068555 1411 raft_consensus.cc:2781] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:11.068763 1411 raft_consensus.cc:686] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:11.068915 1411 consensus_queue.cc:237] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:11.070384 32726 catalog_manager.cc:5484] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:11.212400 1482 raft_consensus.cc:482] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:11.212551 1482 raft_consensus.cc:504] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.212949 1482 leader_election.cc:288] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:11.213435 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2432b3ab36a54fbfa108031f67634e2c" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:11.213451 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2432b3ab36a54fbfa108031f67634e2c" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:11.213599 806 raft_consensus.cc:2454] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:11.213599 408 raft_consensus.cc:2454] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:11.213912 477 leader_election.cc:302] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:11.214088 1482 raft_consensus.cc:2781] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:11.214151 1482 raft_consensus.cc:482] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:11.214179 1482 raft_consensus.cc:3037] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.215126 1482 raft_consensus.cc:504] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.215435 1482 leader_election.cc:288] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:11.215760 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2432b3ab36a54fbfa108031f67634e2c" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:11.215946 408 raft_consensus.cc:3037] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.215960 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2432b3ab36a54fbfa108031f67634e2c" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:11.216077 806 raft_consensus.cc:3037] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.216771 408 raft_consensus.cc:2454] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:11.216847 806 raft_consensus.cc:2454] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:11.217067 476 leader_election.cc:302] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:11.217260 1482 raft_consensus.cc:2781] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:11.217340 1482 raft_consensus.cc:686] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:11.217558 1482 consensus_queue.cc:237] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.219350 32726 catalog_manager.cc:5484] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:11.236701 1425 raft_consensus.cc:482] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:11.236815 1425 raft_consensus.cc:504] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.237246 1425 leader_election.cc:288] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:11.237697 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8c5159a1e24471ab878ffb50a6df05" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:11.237712 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8c5159a1e24471ab878ffb50a6df05" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:11.237838 542 raft_consensus.cc:2454] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:11.237851 408 raft_consensus.cc:2454] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:11.238186 740 leader_election.cc:302] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:11.238377 1425 raft_consensus.cc:2781] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:11.238448 1425 raft_consensus.cc:482] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:11.238502 1425 raft_consensus.cc:3037] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.239310 1425 raft_consensus.cc:504] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.239650 1425 leader_election.cc:288] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:11.239957 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8c5159a1e24471ab878ffb50a6df05" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:11.240010 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8c5159a1e24471ab878ffb50a6df05" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:11.240070 542 raft_consensus.cc:3037] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.240118 408 raft_consensus.cc:3037] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.240831 542 raft_consensus.cc:2454] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:11.241034 408 raft_consensus.cc:2454] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:11.241108 738 leader_election.cc:302] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:11.241284 1425 raft_consensus.cc:2781] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:11.241369 1425 raft_consensus.cc:686] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:11.241503 1425 consensus_queue.cc:237] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.243225 32726 catalog_manager.cc:5484] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:11.363883 1594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:11.364207 1594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:11.367512 1594 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 14:00:11.446019 806 raft_consensus.cc:3037] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.446175 674 raft_consensus.cc:1261] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.446635 1590 consensus_queue.cc:1035] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:11.447311 806 raft_consensus.cc:1261] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.449168 408 raft_consensus.cc:1261] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.449398 542 raft_consensus.cc:1261] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.449769 1406 consensus_queue.cc:1035] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:11.451910 806 raft_consensus.cc:1261] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.452148 409 raft_consensus.cc:1261] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.452311 1409 consensus_queue.cc:1035] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:11.452369 1411 consensus_queue.cc:1035] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:11.457262 1482 consensus_queue.cc:1035] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:11.460597 1406 consensus_queue.cc:1035] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 113.553 ms
time per row: 0.0567766 ms
I20240502 14:00:11.521651 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:11.526921 654 tablet_service.cc:1467] Processing CreateTablet for tablet edcd3af1a04c43d3ab79d539b758ecb0 (DEFAULT_TABLE table=rebalance_test_table_3 [id=673d46c73d60440fb2f914f93a51b362]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.526985 388 tablet_service.cc:1467] Processing CreateTablet for tablet edcd3af1a04c43d3ab79d539b758ecb0 (DEFAULT_TABLE table=rebalance_test_table_3 [id=673d46c73d60440fb2f914f93a51b362]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.527283 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edcd3af1a04c43d3ab79d539b758ecb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.527282 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edcd3af1a04c43d3ab79d539b758ecb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.527307 521 tablet_service.cc:1467] Processing CreateTablet for tablet edcd3af1a04c43d3ab79d539b758ecb0 (DEFAULT_TABLE table=rebalance_test_table_3 [id=673d46c73d60440fb2f914f93a51b362]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.527594 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edcd3af1a04c43d3ab79d539b758ecb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.528761 387 tablet_service.cc:1467] Processing CreateTablet for tablet 95101f5c218746eab5095b984181a5fc (DEFAULT_TABLE table=rebalance_test_table_3 [id=673d46c73d60440fb2f914f93a51b362]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.529017 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95101f5c218746eab5095b984181a5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.529608 1577 tablet_bootstrap.cc:492] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:11.529620 1580 tablet_bootstrap.cc:492] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:11.530413 1577 tablet_bootstrap.cc:654] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.530413 1580 tablet_bootstrap.cc:654] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.530997 1579 tablet_bootstrap.cc:492] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:11.531208 785 tablet_service.cc:1467] Processing CreateTablet for tablet 4701a1452cc3428894e0c3a672768b74 (DEFAULT_TABLE table=rebalance_test_table_3 [id=673d46c73d60440fb2f914f93a51b362]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.531363 653 tablet_service.cc:1467] Processing CreateTablet for tablet 4701a1452cc3428894e0c3a672768b74 (DEFAULT_TABLE table=rebalance_test_table_3 [id=673d46c73d60440fb2f914f93a51b362]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.531493 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4701a1452cc3428894e0c3a672768b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.531627 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4701a1452cc3428894e0c3a672768b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.531796 654 tablet_service.cc:1467] Processing CreateTablet for tablet 95101f5c218746eab5095b984181a5fc (DEFAULT_TABLE table=rebalance_test_table_3 [id=673d46c73d60440fb2f914f93a51b362]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.532042 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95101f5c218746eab5095b984181a5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.532120 784 tablet_service.cc:1467] Processing CreateTablet for tablet 95101f5c218746eab5095b984181a5fc (DEFAULT_TABLE table=rebalance_test_table_3 [id=673d46c73d60440fb2f914f93a51b362]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.532323 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95101f5c218746eab5095b984181a5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.531867 1579 tablet_bootstrap.cc:654] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.533619 1576 tablet_bootstrap.cc:492] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:11.534528 1576 tablet_bootstrap.cc:654] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.535537 1580 tablet_bootstrap.cc:492] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:11.535629 1580 ts_tablet_manager.cc:1367] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 14:00:11.535653 1579 tablet_bootstrap.cc:492] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:11.535728 1579 ts_tablet_manager.cc:1367] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 14:00:11.535868 521 tablet_service.cc:1467] Processing CreateTablet for tablet 4701a1452cc3428894e0c3a672768b74 (DEFAULT_TABLE table=rebalance_test_table_3 [id=673d46c73d60440fb2f914f93a51b362]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.536060 1577 tablet_bootstrap.cc:492] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:11.536094 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4701a1452cc3428894e0c3a672768b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.536145 1577 ts_tablet_manager.cc:1367] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:11.536110 1580 raft_consensus.cc:348] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.536212 1580 raft_consensus.cc:374] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.536166 1579 raft_consensus.cc:348] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.536249 1580 raft_consensus.cc:729] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.536255 1579 raft_consensus.cc:374] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.536288 1579 raft_consensus.cc:729] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.536381 1580 consensus_queue.cc:260] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.536412 1579 consensus_queue.cc:260] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.536574 1577 raft_consensus.cc:348] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.536651 1579 ts_tablet_manager.cc:1398] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.536652 1580 ts_tablet_manager.cc:1398] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.536664 1577 raft_consensus.cc:374] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.536695 1577 raft_consensus.cc:729] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.536799 1579 tablet_bootstrap.cc:492] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:11.536808 1577 consensus_queue.cc:260] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.537077 1577 ts_tablet_manager.cc:1398] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.537253 1577 tablet_bootstrap.cc:492] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:11.537616 1579 tablet_bootstrap.cc:654] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.538792 1580 tablet_bootstrap.cc:492] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:11.539036 1576 tablet_bootstrap.cc:492] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:11.539151 1576 ts_tablet_manager.cc:1367] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:11.539629 1576 raft_consensus.cc:348] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.539767 1576 raft_consensus.cc:374] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.539808 1576 raft_consensus.cc:729] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.539954 1576 consensus_queue.cc:260] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.540633 1576 ts_tablet_manager.cc:1398] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.540803 1576 tablet_bootstrap.cc:492] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:11.541045 1580 tablet_bootstrap.cc:654] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.541518 1577 tablet_bootstrap.cc:654] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.541838 1554 raft_consensus.cc:482] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:11.541931 1554 raft_consensus.cc:504] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.542002 1576 tablet_bootstrap.cc:654] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.542299 1554 leader_election.cc:288] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:11.542783 1579 tablet_bootstrap.cc:492] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:11.542884 1579 ts_tablet_manager.cc:1367] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 14:00:11.543242 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edcd3af1a04c43d3ab79d539b758ecb0" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:11.543367 409 raft_consensus.cc:2454] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:11.543421 1576 tablet_bootstrap.cc:492] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:11.543332 1579 raft_consensus.cc:348] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.543483 1579 raft_consensus.cc:374] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.543519 1579 raft_consensus.cc:729] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.543515 1576 ts_tablet_manager.cc:1367] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:11.543640 1579 consensus_queue.cc:260] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.543923 1579 ts_tablet_manager.cc:1398] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:11.543962 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edcd3af1a04c43d3ab79d539b758ecb0" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:11.544023 1576 raft_consensus.cc:348] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.544085 542 raft_consensus.cc:2454] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:11.544100 1579 tablet_bootstrap.cc:492] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:11.544106 1576 raft_consensus.cc:374] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.544137 1576 raft_consensus.cc:729] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.544253 1576 consensus_queue.cc:260] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.543658 608 leader_election.cc:302] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:11.544644 1576 ts_tablet_manager.cc:1398] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.544732 1554 raft_consensus.cc:2781] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:11.544786 1554 raft_consensus.cc:482] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:11.544828 1554 raft_consensus.cc:3037] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.545085 1579 tablet_bootstrap.cc:654] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.545339 1577 tablet_bootstrap.cc:492] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:11.545439 1577 ts_tablet_manager.cc:1367] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 14:00:11.545835 1554 raft_consensus.cc:504] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.545898 1577 raft_consensus.cc:348] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.545997 1577 raft_consensus.cc:374] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.546029 1577 raft_consensus.cc:729] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.546144 1577 consensus_queue.cc:260] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.546171 1554 leader_election.cc:288] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:11.546461 1577 ts_tablet_manager.cc:1398] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.546732 1580 tablet_bootstrap.cc:492] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:11.546811 1580 ts_tablet_manager.cc:1367] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:11.547139 1579 tablet_bootstrap.cc:492] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:11.547215 1579 ts_tablet_manager.cc:1367] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:11.547251 1580 raft_consensus.cc:348] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.547343 1580 raft_consensus.cc:374] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.547376 1580 raft_consensus.cc:729] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.547485 1580 consensus_queue.cc:260] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.547598 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edcd3af1a04c43d3ab79d539b758ecb0" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:11.547614 1579 raft_consensus.cc:348] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.547685 1579 raft_consensus.cc:374] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.547699 542 raft_consensus.cc:3037] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.547719 1579 raft_consensus.cc:729] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.547784 1580 ts_tablet_manager.cc:1398] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:11.547853 1579 consensus_queue.cc:260] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.548094 1579 ts_tablet_manager.cc:1398] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.548282 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edcd3af1a04c43d3ab79d539b758ecb0" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:11.548390 409 raft_consensus.cc:3037] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.548581 542 raft_consensus.cc:2454] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:11.548918 606 leader_election.cc:302] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:11.549064 1554 raft_consensus.cc:2781] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:11.549135 1554 raft_consensus.cc:686] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:11.549250 1554 consensus_queue.cc:237] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.549378 409 raft_consensus.cc:2454] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:11.552215 32730 catalog_manager.cc:5484] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:11.580271 1409 raft_consensus.cc:482] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:11.580406 1409 raft_consensus.cc:504] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.580803 1409 leader_election.cc:288] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:11.581213 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4701a1452cc3428894e0c3a672768b74" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:11.581219 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4701a1452cc3428894e0c3a672768b74" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:11.581338 805 raft_consensus.cc:2454] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:11.581368 674 raft_consensus.cc:2454] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:11.581635 477 leader_election.cc:302] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:11.581801 1409 raft_consensus.cc:2781] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:11.581862 1409 raft_consensus.cc:482] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:11.581904 1409 raft_consensus.cc:3037] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.582795 1409 raft_consensus.cc:504] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.583128 1409 leader_election.cc:288] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:11.583406 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4701a1452cc3428894e0c3a672768b74" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:11.583523 674 raft_consensus.cc:3037] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.583587 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4701a1452cc3428894e0c3a672768b74" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:11.583688 805 raft_consensus.cc:3037] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.584316 674 raft_consensus.cc:2454] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:11.584509 805 raft_consensus.cc:2454] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:11.584568 474 leader_election.cc:302] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:11.584730 1409 raft_consensus.cc:2781] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:11.584801 1409 raft_consensus.cc:686] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:11.584942 1409 consensus_queue.cc:237] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.586436 32733 catalog_manager.cc:5484] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:11.645215 1554 raft_consensus.cc:482] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:11.645381 1554 raft_consensus.cc:504] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.645898 1554 leader_election.cc:288] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:11.646340 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95101f5c218746eab5095b984181a5fc" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:11.646514 409 raft_consensus.cc:2454] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:11.646901 608 leader_election.cc:302] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:11.647091 1554 raft_consensus.cc:2781] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:11.647157 1554 raft_consensus.cc:482] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:11.647207 1554 raft_consensus.cc:3037] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.648077 1554 raft_consensus.cc:504] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.648409 1554 leader_election.cc:288] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:11.648799 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95101f5c218746eab5095b984181a5fc" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:11.648932 409 raft_consensus.cc:3037] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.649677 409 raft_consensus.cc:2454] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:11.649673 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95101f5c218746eab5095b984181a5fc" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:11.649760 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95101f5c218746eab5095b984181a5fc" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:11.649794 805 raft_consensus.cc:2454] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:11.649967 608 leader_election.cc:302] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:11.650120 1554 raft_consensus.cc:2781] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:11.650218 1554 raft_consensus.cc:686] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:11.650357 1554 consensus_queue.cc:237] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:11.652122 32733 catalog_manager.cc:5484] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:11.770686 1652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:11.770927 1652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:11.773948 1652 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 14:00:11.854687 542 raft_consensus.cc:1261] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.855129 1646 consensus_queue.cc:1035] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:11.857409 409 raft_consensus.cc:1261] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.857807 674 raft_consensus.cc:1261] T 4701a1452cc3428894e0c3a672768b74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.858287 806 raft_consensus.cc:1261] T 4701a1452cc3428894e0c3a672768b74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.858670 1646 consensus_queue.cc:1035] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:11.858775 1409 consensus_queue.cc:1035] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:11.860889 1409 consensus_queue.cc:1035] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:11.863121 805 raft_consensus.cc:3037] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:11.864341 805 raft_consensus.cc:1261] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.864673 408 raft_consensus.cc:1261] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:11.865033 1646 consensus_queue.cc:1035] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:11.865680 1646 consensus_queue.cc:1035] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 113.49 ms
time per row: 0.0567449 ms
I20240502 14:00:11.926890 32734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:11.933542 521 tablet_service.cc:1467] Processing CreateTablet for tablet 2bef2d8b7cbf45548d64bf94560bed3f (DEFAULT_TABLE table=rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.933899 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bef2d8b7cbf45548d64bf94560bed3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.933542 522 tablet_service.cc:1467] Processing CreateTablet for tablet 00e31a2c43dd4f6b841a2c00fe61875e (DEFAULT_TABLE table=rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.934888 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e31a2c43dd4f6b841a2c00fe61875e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.935029 387 tablet_service.cc:1467] Processing CreateTablet for tablet 2bef2d8b7cbf45548d64bf94560bed3f (DEFAULT_TABLE table=rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.934976 386 tablet_service.cc:1467] Processing CreateTablet for tablet 89dc41d04bab4cc09f6c17a5d8524358 (DEFAULT_TABLE table=rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.935952 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bef2d8b7cbf45548d64bf94560bed3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.936338 1580 tablet_bootstrap.cc:492] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:11.936622 388 tablet_service.cc:1467] Processing CreateTablet for tablet 00e31a2c43dd4f6b841a2c00fe61875e (DEFAULT_TABLE table=rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.936936 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e31a2c43dd4f6b841a2c00fe61875e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.937198 1580 tablet_bootstrap.cc:654] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.933012 654 tablet_service.cc:1467] Processing CreateTablet for tablet 2bef2d8b7cbf45548d64bf94560bed3f (DEFAULT_TABLE table=rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.937667 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bef2d8b7cbf45548d64bf94560bed3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.938313 1577 tablet_bootstrap.cc:492] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:11.939337 1577 tablet_bootstrap.cc:654] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.934231 653 tablet_service.cc:1467] Processing CreateTablet for tablet 89dc41d04bab4cc09f6c17a5d8524358 (DEFAULT_TABLE table=rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.940197 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89dc41d04bab4cc09f6c17a5d8524358. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.940383 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89dc41d04bab4cc09f6c17a5d8524358. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.940524 1580 tablet_bootstrap.cc:492] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:11.940603 1580 ts_tablet_manager.cc:1367] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:11.941073 1580 raft_consensus.cc:348] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.941179 1580 raft_consensus.cc:374] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.941210 1580 raft_consensus.cc:729] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.941329 1580 consensus_queue.cc:260] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.941613 1580 ts_tablet_manager.cc:1398] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:11.941787 1580 tablet_bootstrap.cc:492] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:11.942955 1580 tablet_bootstrap.cc:654] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.943010 1577 tablet_bootstrap.cc:492] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:11.943097 1577 ts_tablet_manager.cc:1367] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:11.943547 1577 raft_consensus.cc:348] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.943656 1577 raft_consensus.cc:374] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.943691 1577 raft_consensus.cc:729] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.943814 1577 consensus_queue.cc:260] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.944139 1577 ts_tablet_manager.cc:1398] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.944305 1577 tablet_bootstrap.cc:492] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:11.944656 785 tablet_service.cc:1467] Processing CreateTablet for tablet 89dc41d04bab4cc09f6c17a5d8524358 (DEFAULT_TABLE table=rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.944656 784 tablet_service.cc:1467] Processing CreateTablet for tablet 00e31a2c43dd4f6b841a2c00fe61875e (DEFAULT_TABLE table=rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:11.944979 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e31a2c43dd4f6b841a2c00fe61875e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.945185 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89dc41d04bab4cc09f6c17a5d8524358. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:11.946861 1577 tablet_bootstrap.cc:654] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.947435 1576 tablet_bootstrap.cc:492] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:11.948477 1576 tablet_bootstrap.cc:654] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.948743 1580 tablet_bootstrap.cc:492] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:11.948886 1580 ts_tablet_manager.cc:1367] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.001s
I20240502 14:00:11.949424 1580 raft_consensus.cc:348] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.949548 1580 raft_consensus.cc:374] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.949582 1580 raft_consensus.cc:729] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.949735 1580 consensus_queue.cc:260] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.950091 1580 ts_tablet_manager.cc:1398] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.950846 1577 tablet_bootstrap.cc:492] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:11.950942 1577 ts_tablet_manager.cc:1367] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:11.951305 1577 raft_consensus.cc:348] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.951408 1577 raft_consensus.cc:374] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.951443 1577 raft_consensus.cc:729] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.951560 1577 consensus_queue.cc:260] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.951804 1577 ts_tablet_manager.cc:1398] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.951989 1577 tablet_bootstrap.cc:492] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:11.952940 1577 tablet_bootstrap.cc:654] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.952960 1579 tablet_bootstrap.cc:492] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:11.953955 1579 tablet_bootstrap.cc:654] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.954200 1577 tablet_bootstrap.cc:492] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:11.954290 1577 ts_tablet_manager.cc:1367] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:11.954754 1577 raft_consensus.cc:348] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.954882 1577 raft_consensus.cc:374] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.954918 1577 raft_consensus.cc:729] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.955034 1577 consensus_queue.cc:260] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.955271 1577 ts_tablet_manager.cc:1398] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.955580 1576 tablet_bootstrap.cc:492] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:11.955668 1576 ts_tablet_manager.cc:1367] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:11.956022 1579 tablet_bootstrap.cc:492] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:11.956107 1579 ts_tablet_manager.cc:1367] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:11.956300 1576 raft_consensus.cc:348] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.956436 1576 raft_consensus.cc:374] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.956476 1576 raft_consensus.cc:729] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.956605 1576 consensus_queue.cc:260] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.956935 1579 raft_consensus.cc:348] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.957060 1579 raft_consensus.cc:374] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.957101 1579 raft_consensus.cc:729] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.957226 1579 consensus_queue.cc:260] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.957526 1579 ts_tablet_manager.cc:1398] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:11.957659 1576 ts_tablet_manager.cc:1398] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:11.957702 1579 tablet_bootstrap.cc:492] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:11.957826 1576 tablet_bootstrap.cc:492] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:11.958648 1579 tablet_bootstrap.cc:654] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.959152 1576 tablet_bootstrap.cc:654] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:11.960275 1576 tablet_bootstrap.cc:492] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:11.960340 1579 tablet_bootstrap.cc:492] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:11.960379 1576 ts_tablet_manager.cc:1367] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:11.960453 1579 ts_tablet_manager.cc:1367] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:11.960862 1576 raft_consensus.cc:348] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.960865 1579 raft_consensus.cc:348] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.960973 1579 raft_consensus.cc:374] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.960974 1576 raft_consensus.cc:374] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:11.961011 1579 raft_consensus.cc:729] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.961015 1576 raft_consensus.cc:729] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:11.961117 1576 consensus_queue.cc:260] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:11.961125 1579 consensus_queue.cc:260] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:11.961422 1579 ts_tablet_manager.cc:1398] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:11.961464 1576 ts_tablet_manager.cc:1398] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:12.002231 1425 raft_consensus.cc:482] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:12.002353 1425 raft_consensus.cc:504] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.002748 1425 leader_election.cc:288] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:12.003120 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dc41d04bab4cc09f6c17a5d8524358" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:12.003152 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dc41d04bab4cc09f6c17a5d8524358" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:12.003259 408 raft_consensus.cc:2454] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:12.003284 674 raft_consensus.cc:2454] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:12.003535 740 leader_election.cc:302] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:12.003679 1425 raft_consensus.cc:2781] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:12.003736 1425 raft_consensus.cc:482] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:12.003787 1425 raft_consensus.cc:3037] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.004729 1425 raft_consensus.cc:504] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.005061 1425 leader_election.cc:288] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:12.005355 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dc41d04bab4cc09f6c17a5d8524358" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:12.005393 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dc41d04bab4cc09f6c17a5d8524358" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:12.005491 674 raft_consensus.cc:3037] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.005510 408 raft_consensus.cc:3037] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.006248 408 raft_consensus.cc:2454] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:12.006263 674 raft_consensus.cc:2454] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:12.006528 740 leader_election.cc:302] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:12.006695 1425 raft_consensus.cc:2781] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:12.006785 1425 raft_consensus.cc:686] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:12.006942 1425 consensus_queue.cc:237] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.008562 32727 catalog_manager.cc:5484] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:12.167598 1409 raft_consensus.cc:482] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:12.167723 1409 raft_consensus.cc:504] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.168164 1409 leader_election.cc:288] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:12.168627 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e31a2c43dd4f6b841a2c00fe61875e" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:12.168635 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e31a2c43dd4f6b841a2c00fe61875e" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:12.168764 805 raft_consensus.cc:2454] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:12.168766 408 raft_consensus.cc:2454] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:12.169070 476 leader_election.cc:302] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:12.169232 1409 raft_consensus.cc:2781] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:12.169288 1409 raft_consensus.cc:482] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:12.169333 1409 raft_consensus.cc:3037] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.170097 1409 raft_consensus.cc:504] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.170390 1409 leader_election.cc:288] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:12.170684 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e31a2c43dd4f6b841a2c00fe61875e" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:12.170718 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e31a2c43dd4f6b841a2c00fe61875e" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:12.170800 408 raft_consensus.cc:3037] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.170799 805 raft_consensus.cc:3037] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.171515 408 raft_consensus.cc:2454] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:12.171577 805 raft_consensus.cc:2454] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:12.171774 476 leader_election.cc:302] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:12.171976 1409 raft_consensus.cc:2781] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:12.172057 1409 raft_consensus.cc:686] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:12.172192 1409 consensus_queue.cc:237] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.173704 32727 catalog_manager.cc:5484] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:12.187951 1409 raft_consensus.cc:482] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:12.188053 1409 raft_consensus.cc:504] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.188398 1409 leader_election.cc:288] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:12.188776 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bef2d8b7cbf45548d64bf94560bed3f" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:12.188792 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bef2d8b7cbf45548d64bf94560bed3f" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:12.188901 408 raft_consensus.cc:2454] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:12.188903 674 raft_consensus.cc:2454] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:12.189167 476 leader_election.cc:302] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:12.189328 1409 raft_consensus.cc:2781] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:12.189384 1409 raft_consensus.cc:482] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:12.189435 1409 raft_consensus.cc:3037] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.190176 1409 raft_consensus.cc:504] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.190481 1409 leader_election.cc:288] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:12.190745 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bef2d8b7cbf45548d64bf94560bed3f" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:12.190832 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bef2d8b7cbf45548d64bf94560bed3f" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:12.190869 674 raft_consensus.cc:3037] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.190939 408 raft_consensus.cc:3037] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.191598 674 raft_consensus.cc:2454] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:12.191622 408 raft_consensus.cc:2454] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:12.191907 476 leader_election.cc:302] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:12.192071 1409 raft_consensus.cc:2781] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:12.192152 1409 raft_consensus.cc:686] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:12.192299 1409 consensus_queue.cc:237] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.193904 32727 catalog_manager.cc:5484] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } }
W20240502 14:00:12.348453 1704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:12.348737 1704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:12.351940 1704 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 14:00:12.405320 1425 consensus_queue.cc:1035] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:12.409319 1630 consensus_queue.cc:1035] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:12.445091 408 raft_consensus.cc:1261] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:12.445246 805 raft_consensus.cc:1261] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:12.445571 1409 consensus_queue.cc:1035] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:12.445729 1482 consensus_queue.cc:1035] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:12.452257 406 raft_consensus.cc:1261] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:12.452642 674 raft_consensus.cc:1261] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:12.452942 1482 consensus_queue.cc:1035] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:12.453244 1482 consensus_queue.cc:1035] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 123.534 ms
time per row: 0.0617669 ms
I20240502 14:00:12.519160 32727 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:12.524258 654 tablet_service.cc:1467] Processing CreateTablet for tablet a35103d97b874010b36c1863b2258c33 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4ce1c57cf9054191b787f9697c0c8f88]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.524320 784 tablet_service.cc:1467] Processing CreateTablet for tablet a35103d97b874010b36c1863b2258c33 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4ce1c57cf9054191b787f9697c0c8f88]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.524510 522 tablet_service.cc:1467] Processing CreateTablet for tablet a35103d97b874010b36c1863b2258c33 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4ce1c57cf9054191b787f9697c0c8f88]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.524596 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35103d97b874010b36c1863b2258c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.524605 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35103d97b874010b36c1863b2258c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.524750 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35103d97b874010b36c1863b2258c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.525301 386 tablet_service.cc:1467] Processing CreateTablet for tablet 3f9b1351891d4690b67f8eef000a902b (DEFAULT_TABLE table=rebalance_test_table_5 [id=4ce1c57cf9054191b787f9697c0c8f88]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.525559 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f9b1351891d4690b67f8eef000a902b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.527297 522 tablet_service.cc:1467] Processing CreateTablet for tablet 556b3a4cab9245d4939ba4c10bf9d4b4 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4ce1c57cf9054191b787f9697c0c8f88]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.527392 1770 tablet_bootstrap.cc:492] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:12.527451 521 tablet_service.cc:1467] Processing CreateTablet for tablet 3f9b1351891d4690b67f8eef000a902b (DEFAULT_TABLE table=rebalance_test_table_5 [id=4ce1c57cf9054191b787f9697c0c8f88]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.527551 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556b3a4cab9245d4939ba4c10bf9d4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.527647 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f9b1351891d4690b67f8eef000a902b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.528081 653 tablet_service.cc:1467] Processing CreateTablet for tablet 3f9b1351891d4690b67f8eef000a902b (DEFAULT_TABLE table=rebalance_test_table_5 [id=4ce1c57cf9054191b787f9697c0c8f88]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.528262 1770 tablet_bootstrap.cc:654] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.528322 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f9b1351891d4690b67f8eef000a902b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.528853 1771 tablet_bootstrap.cc:492] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:12.529011 784 tablet_service.cc:1467] Processing CreateTablet for tablet 556b3a4cab9245d4939ba4c10bf9d4b4 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4ce1c57cf9054191b787f9697c0c8f88]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.529253 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556b3a4cab9245d4939ba4c10bf9d4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.529724 1771 tablet_bootstrap.cc:654] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.530755 1770 tablet_bootstrap.cc:492] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:12.530853 1770 ts_tablet_manager.cc:1367] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:12.531270 1770 raft_consensus.cc:348] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.531387 1770 raft_consensus.cc:374] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:12.531419 1770 raft_consensus.cc:729] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:12.531582 1769 tablet_bootstrap.cc:492] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:12.531581 1770 consensus_queue.cc:260] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.531878 1770 ts_tablet_manager.cc:1398] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:12.532054 1770 tablet_bootstrap.cc:492] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:12.532464 654 tablet_service.cc:1467] Processing CreateTablet for tablet 556b3a4cab9245d4939ba4c10bf9d4b4 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4ce1c57cf9054191b787f9697c0c8f88]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.532557 1769 tablet_bootstrap.cc:654] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.532677 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556b3a4cab9245d4939ba4c10bf9d4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.532828 1770 tablet_bootstrap.cc:654] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.534085 1771 tablet_bootstrap.cc:492] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:12.534174 1771 ts_tablet_manager.cc:1367] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:12.534307 1770 tablet_bootstrap.cc:492] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:12.534662 1771 raft_consensus.cc:348] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.534754 1772 tablet_bootstrap.cc:492] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:12.534772 1771 raft_consensus.cc:374] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:12.534828 1771 raft_consensus.cc:729] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:12.534972 1771 consensus_queue.cc:260] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.534384 1770 ts_tablet_manager.cc:1367] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:12.535548 1770 raft_consensus.cc:348] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.535646 1770 raft_consensus.cc:374] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:12.535669 1772 tablet_bootstrap.cc:654] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.535681 1770 raft_consensus.cc:729] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:12.535806 1770 consensus_queue.cc:260] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.536195 1770 ts_tablet_manager.cc:1398] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:12.536377 1770 tablet_bootstrap.cc:492] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:12.536685 1772 tablet_bootstrap.cc:492] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:12.536777 1772 ts_tablet_manager.cc:1367] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:12.537251 1770 tablet_bootstrap.cc:654] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.537182 1772 raft_consensus.cc:348] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:12.537294 1772 raft_consensus.cc:374] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:12.537326 1772 raft_consensus.cc:729] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:12.537451 1772 consensus_queue.cc:260] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:12.537734 1772 ts_tablet_manager.cc:1398] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:12.537897 1771 ts_tablet_manager.cc:1398] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:12.538054 1771 tablet_bootstrap.cc:492] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:12.538465 1769 tablet_bootstrap.cc:492] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:12.538550 1769 ts_tablet_manager.cc:1367] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:00:12.539009 1771 tablet_bootstrap.cc:654] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.539044 1770 tablet_bootstrap.cc:492] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:12.538983 1769 raft_consensus.cc:348] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.539083 1769 raft_consensus.cc:374] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:12.539116 1769 raft_consensus.cc:729] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:12.539147 1770 ts_tablet_manager.cc:1367] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:00:12.539212 1769 consensus_queue.cc:260] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.539479 1769 ts_tablet_manager.cc:1398] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:12.539669 1769 tablet_bootstrap.cc:492] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:12.539752 1770 raft_consensus.cc:348] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:12.540000 1770 raft_consensus.cc:374] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:12.540046 1770 raft_consensus.cc:729] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:12.540163 1770 consensus_queue.cc:260] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:12.540442 1770 ts_tablet_manager.cc:1398] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:12.540664 1769 tablet_bootstrap.cc:654] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.541817 1769 tablet_bootstrap.cc:492] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:12.541900 1769 ts_tablet_manager.cc:1367] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:12.543033 1771 tablet_bootstrap.cc:492] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:12.542454 1769 raft_consensus.cc:348] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:12.543089 1769 raft_consensus.cc:374] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:12.543134 1769 raft_consensus.cc:729] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:12.543246 1769 consensus_queue.cc:260] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:12.543411 1771 ts_tablet_manager.cc:1367] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:12.543555 1769 ts_tablet_manager.cc:1398] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:12.543768 1769 tablet_bootstrap.cc:492] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:12.544034 1771 raft_consensus.cc:348] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.544207 1771 raft_consensus.cc:374] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:12.544261 1771 raft_consensus.cc:729] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:12.544410 1771 consensus_queue.cc:260] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.544704 1771 ts_tablet_manager.cc:1398] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:12.544752 1769 tablet_bootstrap.cc:654] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.545996 1769 tablet_bootstrap.cc:492] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:12.546103 1769 ts_tablet_manager.cc:1367] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:12.546545 1769 raft_consensus.cc:348] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.546663 1769 raft_consensus.cc:374] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:12.546706 1769 raft_consensus.cc:729] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:12.546828 1769 consensus_queue.cc:260] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.547081 1769 ts_tablet_manager.cc:1398] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:12.570555 1691 raft_consensus.cc:482] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:12.570660 1691 raft_consensus.cc:504] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.571049 1691 leader_election.cc:288] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:12.571399 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35103d97b874010b36c1863b2258c33" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:12.571410 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35103d97b874010b36c1863b2258c33" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:12.571528 805 raft_consensus.cc:2454] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:12.571550 542 raft_consensus.cc:2454] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:12.571846 609 leader_election.cc:302] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:12.572007 1691 raft_consensus.cc:2781] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:12.572068 1691 raft_consensus.cc:482] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:12.572098 1691 raft_consensus.cc:3037] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.572912 1691 raft_consensus.cc:504] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.573225 1691 leader_election.cc:288] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:12.573535 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35103d97b874010b36c1863b2258c33" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:12.573590 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35103d97b874010b36c1863b2258c33" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:12.573648 805 raft_consensus.cc:3037] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.573714 542 raft_consensus.cc:3037] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.574420 542 raft_consensus.cc:2454] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:12.574419 805 raft_consensus.cc:2454] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:12.574690 606 leader_election.cc:302] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:12.574915 1691 raft_consensus.cc:2781] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:12.574990 1691 raft_consensus.cc:686] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:12.575125 1691 consensus_queue.cc:237] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:12.576766 32734 catalog_manager.cc:5484] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:12.608256 1691 raft_consensus.cc:482] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:12.608379 1691 raft_consensus.cc:504] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:12.608851 1691 leader_election.cc:288] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:12.609265 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9b1351891d4690b67f8eef000a902b" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:12.609251 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9b1351891d4690b67f8eef000a902b" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:12.609409 542 raft_consensus.cc:2454] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:12.609409 409 raft_consensus.cc:2454] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:12.609721 608 leader_election.cc:302] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:12.609908 1691 raft_consensus.cc:2781] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:12.609987 1691 raft_consensus.cc:482] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:12.610033 1691 raft_consensus.cc:3037] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.610793 1691 raft_consensus.cc:504] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:12.611141 1691 leader_election.cc:288] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:12.611418 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9b1351891d4690b67f8eef000a902b" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:12.611454 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9b1351891d4690b67f8eef000a902b" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:12.611527 542 raft_consensus.cc:3037] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.611594 409 raft_consensus.cc:3037] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.612322 542 raft_consensus.cc:2454] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:12.612394 409 raft_consensus.cc:2454] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:12.612604 606 leader_election.cc:302] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:12.612764 1691 raft_consensus.cc:2781] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:12.612854 1691 raft_consensus.cc:686] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:12.613006 1691 consensus_queue.cc:237] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:12.614722 32734 catalog_manager.cc:5484] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:12.701449 1746 raft_consensus.cc:482] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:12.701609 1746 raft_consensus.cc:504] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.702026 1746 leader_election.cc:288] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:12.702504 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556b3a4cab9245d4939ba4c10bf9d4b4" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:12.702510 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556b3a4cab9245d4939ba4c10bf9d4b4" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:12.702641 805 raft_consensus.cc:2454] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:12.702647 674 raft_consensus.cc:2454] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:12.702951 477 leader_election.cc:302] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:12.703128 1746 raft_consensus.cc:2781] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:12.703181 1746 raft_consensus.cc:482] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:12.703205 1746 raft_consensus.cc:3037] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.704151 1746 raft_consensus.cc:504] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.704509 1746 leader_election.cc:288] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:12.704828 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556b3a4cab9245d4939ba4c10bf9d4b4" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:12.704849 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556b3a4cab9245d4939ba4c10bf9d4b4" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:12.704946 674 raft_consensus.cc:3037] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.704950 805 raft_consensus.cc:3037] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:12.705667 674 raft_consensus.cc:2454] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:12.705691 805 raft_consensus.cc:2454] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:12.705917 474 leader_election.cc:302] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:12.706092 1746 raft_consensus.cc:2781] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:12.706187 1746 raft_consensus.cc:686] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:12.706342 1746 consensus_queue.cc:237] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.708099 32734 catalog_manager.cc:5484] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:12.827549 1785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:12.827782 1785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:12.830931 1785 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 14:00:12.919075 542 raft_consensus.cc:1261] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:12.919523 409 raft_consensus.cc:1261] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:12.919517 1687 consensus_queue.cc:1035] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:12.919909 1687 consensus_queue.cc:1035] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:12.924239 674 raft_consensus.cc:1261] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:12.924607 805 raft_consensus.cc:1261] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:12.925055 1746 consensus_queue.cc:1035] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:12.925225 805 raft_consensus.cc:1261] T a35103d97b874010b36c1863b2258c33 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:12.925653 1691 consensus_queue.cc:1035] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:12.925884 541 raft_consensus.cc:1261] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:12.925963 1746 consensus_queue.cc:1035] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:12.929003 1691 consensus_queue.cc:1035] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:12.945210 1741 mvcc.cc:205] Tried to move back new op lower bound from 7023240859311783936 to 7023240858064285696. Current Snapshot: MvccSnapshot[applied={T|T < 7023240859311783936}]
I20240502 14:00:12.956737 1628 mvcc.cc:205] Tried to move back new op lower bound from 7023240859322327040 to 7023240858446499840. Current Snapshot: MvccSnapshot[applied={T|T < 7023240859322327040}]
INSERT report
rows total: 2000
time total: 112.967 ms
time per row: 0.0564833 ms
I20240502 14:00:12.986874 32726 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:12.992406 386 tablet_service.cc:1467] Processing CreateTablet for tablet a3e7aa78b21a42f89cd3ac29cd113206 (DEFAULT_TABLE table=rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.992487 784 tablet_service.cc:1467] Processing CreateTablet for tablet a3e7aa78b21a42f89cd3ac29cd113206 (DEFAULT_TABLE table=rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.992754 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3e7aa78b21a42f89cd3ac29cd113206. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.992810 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3e7aa78b21a42f89cd3ac29cd113206. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.992972 785 tablet_service.cc:1467] Processing CreateTablet for tablet 8122e08018cb40e987cdd4e9053c6cbd (DEFAULT_TABLE table=rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.993193 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8122e08018cb40e987cdd4e9053c6cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.993656 521 tablet_service.cc:1467] Processing CreateTablet for tablet a3e7aa78b21a42f89cd3ac29cd113206 (DEFAULT_TABLE table=rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.993971 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3e7aa78b21a42f89cd3ac29cd113206. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.994210 654 tablet_service.cc:1467] Processing CreateTablet for tablet 8122e08018cb40e987cdd4e9053c6cbd (DEFAULT_TABLE table=rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.994455 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8122e08018cb40e987cdd4e9053c6cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.995324 387 tablet_service.cc:1467] Processing CreateTablet for tablet 6b96ddb6ec9f4ef0bda0d354fc39adec (DEFAULT_TABLE table=rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.995324 785 tablet_service.cc:1467] Processing CreateTablet for tablet 6b96ddb6ec9f4ef0bda0d354fc39adec (DEFAULT_TABLE table=rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.995538 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b96ddb6ec9f4ef0bda0d354fc39adec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.995546 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b96ddb6ec9f4ef0bda0d354fc39adec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.996201 1771 tablet_bootstrap.cc:492] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:12.996832 388 tablet_service.cc:1467] Processing CreateTablet for tablet 8122e08018cb40e987cdd4e9053c6cbd (DEFAULT_TABLE table=rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.997079 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8122e08018cb40e987cdd4e9053c6cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.997381 1771 tablet_bootstrap.cc:654] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.997399 1769 tablet_bootstrap.cc:492] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:12.998035 1770 tablet_bootstrap.cc:492] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:12.998284 1769 tablet_bootstrap.cc:654] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.998435 1771 tablet_bootstrap.cc:492] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:12.998497 1771 ts_tablet_manager.cc:1367] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:12.998821 1771 raft_consensus.cc:348] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.993656 522 tablet_service.cc:1467] Processing CreateTablet for tablet 6b96ddb6ec9f4ef0bda0d354fc39adec (DEFAULT_TABLE table=rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:12.998898 1771 raft_consensus.cc:374] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:12.998919 1771 raft_consensus.cc:729] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:12.998958 1770 tablet_bootstrap.cc:654] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:12.999071 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b96ddb6ec9f4ef0bda0d354fc39adec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:12.999033 1771 consensus_queue.cc:260] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:12.999305 1771 ts_tablet_manager.cc:1398] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:12.999461 1771 tablet_bootstrap.cc:492] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:13.000272 1771 tablet_bootstrap.cc:654] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.001396 1770 tablet_bootstrap.cc:492] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:13.001493 1770 ts_tablet_manager.cc:1367] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:13.001770 1771 tablet_bootstrap.cc:492] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:13.001850 1771 ts_tablet_manager.cc:1367] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:13.001928 1770 raft_consensus.cc:348] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.002039 1770 raft_consensus.cc:374] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.002077 1770 raft_consensus.cc:729] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.002198 1770 consensus_queue.cc:260] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.002244 1771 raft_consensus.cc:348] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.002343 1771 raft_consensus.cc:374] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.002388 1771 raft_consensus.cc:729] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.002485 1770 ts_tablet_manager.cc:1398] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:13.002519 1771 consensus_queue.cc:260] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.002664 1770 tablet_bootstrap.cc:492] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:13.002784 1771 ts_tablet_manager.cc:1398] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:13.003312 1771 tablet_bootstrap.cc:492] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:13.003451 1769 tablet_bootstrap.cc:492] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:13.003561 1769 ts_tablet_manager.cc:1367] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:13.003598 1770 tablet_bootstrap.cc:654] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.004096 1769 raft_consensus.cc:348] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.004169 1771 tablet_bootstrap.cc:654] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.004218 1769 raft_consensus.cc:374] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.004261 1769 raft_consensus.cc:729] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.004377 1769 consensus_queue.cc:260] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.004649 1769 ts_tablet_manager.cc:1398] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:13.005178 1770 tablet_bootstrap.cc:492] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:13.005268 1770 ts_tablet_manager.cc:1367] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:13.005671 1771 tablet_bootstrap.cc:492] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:13.005759 1771 ts_tablet_manager.cc:1367] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:13.005908 1770 raft_consensus.cc:348] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.005997 1770 raft_consensus.cc:374] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.006034 1770 raft_consensus.cc:729] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.006165 1770 consensus_queue.cc:260] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.006512 1770 ts_tablet_manager.cc:1398] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:13.006844 1771 raft_consensus.cc:348] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.006973 1771 raft_consensus.cc:374] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.007006 1771 raft_consensus.cc:729] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.007140 1771 consensus_queue.cc:260] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.009037 1771 ts_tablet_manager.cc:1398] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:13.013481 1772 tablet_bootstrap.cc:492] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:13.014410 1772 tablet_bootstrap.cc:654] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.015316 1772 tablet_bootstrap.cc:492] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:13.015391 1772 ts_tablet_manager.cc:1367] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:13.015739 1772 raft_consensus.cc:348] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.015847 1772 raft_consensus.cc:374] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.015888 1772 raft_consensus.cc:729] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.015995 1772 consensus_queue.cc:260] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.016232 1772 ts_tablet_manager.cc:1398] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:13.016392 1772 tablet_bootstrap.cc:492] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:13.017102 1772 tablet_bootstrap.cc:654] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.018077 1772 tablet_bootstrap.cc:492] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:13.018168 1772 ts_tablet_manager.cc:1367] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:13.018510 1772 raft_consensus.cc:348] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.018595 1772 raft_consensus.cc:374] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.018627 1772 raft_consensus.cc:729] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.018752 1772 consensus_queue.cc:260] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.019008 1772 ts_tablet_manager.cc:1398] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:13.019170 1772 tablet_bootstrap.cc:492] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:13.019974 1772 tablet_bootstrap.cc:654] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.021221 1772 tablet_bootstrap.cc:492] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:13.021288 1772 ts_tablet_manager.cc:1367] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:13.021587 1772 raft_consensus.cc:348] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.021663 1772 raft_consensus.cc:374] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.021687 1772 raft_consensus.cc:729] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.021776 1772 consensus_queue.cc:260] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.022001 1772 ts_tablet_manager.cc:1398] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:13.025838 1820 raft_consensus.cc:482] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:13.025962 1820 raft_consensus.cc:504] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.026367 1820 leader_election.cc:288] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:13.026644 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8122e08018cb40e987cdd4e9053c6cbd" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:13.026732 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8122e08018cb40e987cdd4e9053c6cbd" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:13.026765 408 raft_consensus.cc:2454] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:13.026854 806 raft_consensus.cc:2454] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:13.027086 608 leader_election.cc:302] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:13.027251 1820 raft_consensus.cc:2781] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:13.027302 1820 raft_consensus.cc:482] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:13.027338 1820 raft_consensus.cc:3037] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.028614 1820 raft_consensus.cc:504] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.028941 1820 leader_election.cc:288] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:13.029207 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8122e08018cb40e987cdd4e9053c6cbd" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:13.029264 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8122e08018cb40e987cdd4e9053c6cbd" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:13.029304 408 raft_consensus.cc:3037] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.029364 806 raft_consensus.cc:3037] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.030217 408 raft_consensus.cc:2454] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:13.030217 806 raft_consensus.cc:2454] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:13.030510 609 leader_election.cc:302] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:13.030714 1820 raft_consensus.cc:2781] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:13.030791 1820 raft_consensus.cc:686] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:13.030922 1820 consensus_queue.cc:237] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.032589 32729 catalog_manager.cc:5484] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:13.042790 1751 raft_consensus.cc:482] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:13.042901 1751 raft_consensus.cc:504] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.043342 1751 leader_election.cc:288] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:13.043617 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b96ddb6ec9f4ef0bda0d354fc39adec" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:13.043745 806 raft_consensus.cc:2454] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:13.043841 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b96ddb6ec9f4ef0bda0d354fc39adec" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:13.043977 542 raft_consensus.cc:2454] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:13.044158 344 leader_election.cc:302] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:13.044361 1751 raft_consensus.cc:2781] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:13.044427 1751 raft_consensus.cc:482] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:13.044466 1751 raft_consensus.cc:3037] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.045213 1751 raft_consensus.cc:504] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.045509 1751 leader_election.cc:288] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:13.045833 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b96ddb6ec9f4ef0bda0d354fc39adec" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:13.045853 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b96ddb6ec9f4ef0bda0d354fc39adec" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:13.045953 542 raft_consensus.cc:3037] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.045956 806 raft_consensus.cc:3037] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.046650 806 raft_consensus.cc:2454] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:13.046669 542 raft_consensus.cc:2454] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:13.046914 344 leader_election.cc:302] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:13.047080 1751 raft_consensus.cc:2781] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:13.047168 1751 raft_consensus.cc:686] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:13.047317 1751 consensus_queue.cc:237] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.049054 32729 catalog_manager.cc:5484] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:13.328092 1746 raft_consensus.cc:482] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:13.328245 1746 raft_consensus.cc:504] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.328686 1746 leader_election.cc:288] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:13.329159 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3e7aa78b21a42f89cd3ac29cd113206" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:13.329175 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3e7aa78b21a42f89cd3ac29cd113206" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:13.329299 408 raft_consensus.cc:2454] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:13.329313 806 raft_consensus.cc:2454] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:13.329600 476 leader_election.cc:302] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:13.329771 1746 raft_consensus.cc:2781] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:13.329830 1746 raft_consensus.cc:482] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:13.329883 1746 raft_consensus.cc:3037] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.330760 1746 raft_consensus.cc:504] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.331081 1746 leader_election.cc:288] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:13.331418 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3e7aa78b21a42f89cd3ac29cd113206" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:13.331418 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3e7aa78b21a42f89cd3ac29cd113206" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:13.331528 408 raft_consensus.cc:3037] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.331537 806 raft_consensus.cc:3037] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.332296 408 raft_consensus.cc:2454] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:13.332422 806 raft_consensus.cc:2454] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:13.332614 476 leader_election.cc:302] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:13.332829 1746 raft_consensus.cc:2781] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:13.332911 1746 raft_consensus.cc:686] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:13.333056 1746 consensus_queue.cc:237] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.334833 32729 catalog_manager.cc:5484] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:13.455112 1751 consensus_queue.cc:1035] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:13.459306 1837 consensus_queue.cc:1035] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 14:00:13.478557 1835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:13.478842 1835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:13.482267 1835 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 14:00:13.506697 1820 consensus_queue.cc:1035] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:13.510272 1820 consensus_queue.cc:1035] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:13.581485 408 raft_consensus.cc:1261] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:13.581904 1746 consensus_queue.cc:1035] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:13.582620 805 raft_consensus.cc:1261] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:13.585075 1746 consensus_queue.cc:1035] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:13.609113 1882 mvcc.cc:205] Tried to move back new op lower bound from 7023240862024294400 to 7023240861013450752. Current Snapshot: MvccSnapshot[applied={T|T < 7023240862024294400}]
INSERT report
rows total: 2000
time total: 112.781 ms
time per row: 0.0563903 ms
I20240502 14:00:13.639488 32726 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:13.644793 387 tablet_service.cc:1467] Processing CreateTablet for tablet ee25d04c50a64ab98282e3659c94bd71 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3dec3eec8fda4142913bf2d106d579b1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:13.644866 654 tablet_service.cc:1467] Processing CreateTablet for tablet ee25d04c50a64ab98282e3659c94bd71 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3dec3eec8fda4142913bf2d106d579b1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:13.644973 522 tablet_service.cc:1467] Processing CreateTablet for tablet ee25d04c50a64ab98282e3659c94bd71 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3dec3eec8fda4142913bf2d106d579b1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:13.645144 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee25d04c50a64ab98282e3659c94bd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:13.645164 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee25d04c50a64ab98282e3659c94bd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:13.645205 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee25d04c50a64ab98282e3659c94bd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:13.647156 1898 tablet_bootstrap.cc:492] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:13.647722 387 tablet_service.cc:1467] Processing CreateTablet for tablet 08f30aab835c4380b8b7a01400e5aa55 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3dec3eec8fda4142913bf2d106d579b1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:13.647990 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f30aab835c4380b8b7a01400e5aa55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:13.648113 785 tablet_service.cc:1467] Processing CreateTablet for tablet 08f30aab835c4380b8b7a01400e5aa55 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3dec3eec8fda4142913bf2d106d579b1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:13.648263 522 tablet_service.cc:1467] Processing CreateTablet for tablet 1415dfd1daed4e1a8a00439c77185a55 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3dec3eec8fda4142913bf2d106d579b1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:13.648365 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f30aab835c4380b8b7a01400e5aa55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:13.648476 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1415dfd1daed4e1a8a00439c77185a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:13.648507 1898 tablet_bootstrap.cc:654] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.649812 784 tablet_service.cc:1467] Processing CreateTablet for tablet 1415dfd1daed4e1a8a00439c77185a55 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3dec3eec8fda4142913bf2d106d579b1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:13.649989 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1415dfd1daed4e1a8a00439c77185a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:13.650278 1899 tablet_bootstrap.cc:492] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:13.650956 1902 tablet_bootstrap.cc:492] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:13.651142 1899 tablet_bootstrap.cc:654] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.651473 653 tablet_service.cc:1467] Processing CreateTablet for tablet 1415dfd1daed4e1a8a00439c77185a55 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3dec3eec8fda4142913bf2d106d579b1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:13.651696 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1415dfd1daed4e1a8a00439c77185a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:13.651731 1902 tablet_bootstrap.cc:654] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.652715 1898 tablet_bootstrap.cc:492] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:13.652804 1898 ts_tablet_manager.cc:1367] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:13.653188 1898 raft_consensus.cc:348] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.653278 1898 raft_consensus.cc:374] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.653316 1898 raft_consensus.cc:729] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.653496 1898 consensus_queue.cc:260] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.653847 1898 ts_tablet_manager.cc:1398] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:13.654058 1898 tablet_bootstrap.cc:492] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:13.654067 654 tablet_service.cc:1467] Processing CreateTablet for tablet 08f30aab835c4380b8b7a01400e5aa55 (DEFAULT_TABLE table=rebalance_test_table_7 [id=3dec3eec8fda4142913bf2d106d579b1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:13.654261 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f30aab835c4380b8b7a01400e5aa55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:13.656069 1898 tablet_bootstrap.cc:654] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.656380 1900 tablet_bootstrap.cc:492] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:13.657191 1900 tablet_bootstrap.cc:654] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.657889 1898 tablet_bootstrap.cc:492] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:13.657984 1898 ts_tablet_manager.cc:1367] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:13.658174 1900 tablet_bootstrap.cc:492] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:13.658262 1900 ts_tablet_manager.cc:1367] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:13.658614 1898 raft_consensus.cc:348] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:13.658607 1900 raft_consensus.cc:348] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.658702 1898 raft_consensus.cc:374] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.658736 1898 raft_consensus.cc:729] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.658700 1900 raft_consensus.cc:374] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.658914 1900 raft_consensus.cc:729] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.658897 1898 consensus_queue.cc:260] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:13.659085 1900 consensus_queue.cc:260] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.659451 1900 ts_tablet_manager.cc:1398] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:13.659722 1898 ts_tablet_manager.cc:1398] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:13.659917 1899 tablet_bootstrap.cc:492] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:13.660003 1899 ts_tablet_manager.cc:1367] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:13.660375 1899 raft_consensus.cc:348] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.660425 1900 tablet_bootstrap.cc:492] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:13.660457 1899 raft_consensus.cc:374] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.660485 1899 raft_consensus.cc:729] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.660585 1899 consensus_queue.cc:260] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.660802 1899 ts_tablet_manager.cc:1398] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:13.660934 1899 tablet_bootstrap.cc:492] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:13.661222 1900 tablet_bootstrap.cc:654] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.661742 1899 tablet_bootstrap.cc:654] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.663925 1899 tablet_bootstrap.cc:492] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:13.663981 1902 tablet_bootstrap.cc:492] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:13.664024 1899 ts_tablet_manager.cc:1367] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:13.664161 1902 ts_tablet_manager.cc:1367] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 14:00:13.664480 1899 raft_consensus.cc:348] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.664578 1899 raft_consensus.cc:374] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.664625 1899 raft_consensus.cc:729] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.664732 1899 consensus_queue.cc:260] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.664793 1902 raft_consensus.cc:348] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:13.664896 1902 raft_consensus.cc:374] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.664938 1902 raft_consensus.cc:729] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.665031 1899 ts_tablet_manager.cc:1398] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:13.665066 1902 consensus_queue.cc:260] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:13.665299 1900 tablet_bootstrap.cc:492] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:13.665374 1900 ts_tablet_manager.cc:1367] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:13.665406 1902 ts_tablet_manager.cc:1398] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:13.665565 1902 tablet_bootstrap.cc:492] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:13.665763 1900 raft_consensus.cc:348] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.665861 1900 raft_consensus.cc:374] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.665900 1900 raft_consensus.cc:729] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.666023 1900 consensus_queue.cc:260] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.666273 1900 ts_tablet_manager.cc:1398] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:13.666416 1900 tablet_bootstrap.cc:492] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:13.667148 1900 tablet_bootstrap.cc:654] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.667289 1902 tablet_bootstrap.cc:654] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:13.668216 1900 tablet_bootstrap.cc:492] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:13.669260 1900 ts_tablet_manager.cc:1367] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:13.669665 1900 raft_consensus.cc:348] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:13.669778 1900 raft_consensus.cc:374] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.669867 1900 raft_consensus.cc:729] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.670003 1900 consensus_queue.cc:260] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:13.670269 1900 ts_tablet_manager.cc:1398] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:13.670915 1902 tablet_bootstrap.cc:492] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:13.671044 1902 ts_tablet_manager.cc:1367] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 14:00:13.671458 1902 raft_consensus.cc:348] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.671559 1902 raft_consensus.cc:374] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:13.671586 1902 raft_consensus.cc:729] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:13.671717 1902 consensus_queue.cc:260] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.672056 1902 ts_tablet_manager.cc:1398] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:13.718503 1819 raft_consensus.cc:482] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:13.718626 1819 raft_consensus.cc:504] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:13.719018 1819 leader_election.cc:288] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:13.719322 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f30aab835c4380b8b7a01400e5aa55" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:13.719453 805 raft_consensus.cc:2454] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:13.719404 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f30aab835c4380b8b7a01400e5aa55" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:13.719559 408 raft_consensus.cc:2454] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:13.719732 609 leader_election.cc:302] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:13.719925 1819 raft_consensus.cc:2781] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:13.719991 1819 raft_consensus.cc:482] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:13.720043 1819 raft_consensus.cc:3037] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.720857 1819 raft_consensus.cc:504] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:13.721175 1819 leader_election.cc:288] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:13.721465 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f30aab835c4380b8b7a01400e5aa55" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:13.721491 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f30aab835c4380b8b7a01400e5aa55" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:13.721583 408 raft_consensus.cc:3037] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.721583 805 raft_consensus.cc:3037] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.722272 408 raft_consensus.cc:2454] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:13.722327 805 raft_consensus.cc:2454] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:13.722544 608 leader_election.cc:302] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:13.722699 1819 raft_consensus.cc:2781] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:13.722786 1819 raft_consensus.cc:686] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:13.722923 1819 consensus_queue.cc:237] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:13.724699 32726 catalog_manager.cc:5484] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:13.769829 1747 raft_consensus.cc:482] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:13.769953 1747 raft_consensus.cc:504] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.770351 1747 leader_election.cc:288] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:13.770701 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee25d04c50a64ab98282e3659c94bd71" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:13.770740 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee25d04c50a64ab98282e3659c94bd71" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:13.770857 674 raft_consensus.cc:2454] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:13.770886 408 raft_consensus.cc:2454] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:13.771198 474 leader_election.cc:302] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:13.771412 1747 raft_consensus.cc:2781] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:13.771490 1747 raft_consensus.cc:482] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:13.771546 1747 raft_consensus.cc:3037] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.772406 1747 raft_consensus.cc:504] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.772724 1747 leader_election.cc:288] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:13.773069 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee25d04c50a64ab98282e3659c94bd71" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:13.773082 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee25d04c50a64ab98282e3659c94bd71" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:13.773180 408 raft_consensus.cc:3037] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.773185 674 raft_consensus.cc:3037] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.773907 674 raft_consensus.cc:2454] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:13.773916 408 raft_consensus.cc:2454] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:13.774189 476 leader_election.cc:302] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:13.774343 1747 raft_consensus.cc:2781] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:13.774433 1747 raft_consensus.cc:686] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:13.774581 1747 consensus_queue.cc:237] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:13.776389 32726 catalog_manager.cc:5484] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:13.898635 1747 raft_consensus.cc:482] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:13.898778 1747 raft_consensus.cc:504] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.899227 1747 leader_election.cc:288] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:13.899626 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1415dfd1daed4e1a8a00439c77185a55" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:13.899648 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1415dfd1daed4e1a8a00439c77185a55" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:13.899749 674 raft_consensus.cc:2454] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:13.899796 805 raft_consensus.cc:2454] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:13.900084 474 leader_election.cc:302] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:13.900249 1747 raft_consensus.cc:2781] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:13.900310 1747 raft_consensus.cc:482] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:13.900352 1747 raft_consensus.cc:3037] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.901154 1747 raft_consensus.cc:504] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.901463 1747 leader_election.cc:288] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:13.901789 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1415dfd1daed4e1a8a00439c77185a55" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:13.901789 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1415dfd1daed4e1a8a00439c77185a55" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:13.901913 805 raft_consensus.cc:3037] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.901912 674 raft_consensus.cc:3037] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:13.902829 805 raft_consensus.cc:2454] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:13.902829 674 raft_consensus.cc:2454] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:13.903115 477 leader_election.cc:302] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:13.903261 1747 raft_consensus.cc:2781] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:13.903350 1747 raft_consensus.cc:686] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:13.903501 1747 consensus_queue.cc:237] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:13.905422 32726 catalog_manager.cc:5484] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:14.053681 1914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:14.053957 1914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:14.057152 1914 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240502 14:00:14.106817 1819 consensus_queue.cc:1035] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:14.113649 1856 consensus_queue.cc:1035] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:14.150658 674 raft_consensus.cc:1261] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:14.151068 408 raft_consensus.cc:1261] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:14.152479 1747 consensus_queue.cc:1035] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:14.154408 805 raft_consensus.cc:1261] T 1415dfd1daed4e1a8a00439c77185a55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:14.154923 1746 consensus_queue.cc:1035] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:14.155210 1746 consensus_queue.cc:1035] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:14.163401 673 raft_consensus.cc:1261] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:14.163923 1746 consensus_queue.cc:1035] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:14.184338 1958 mvcc.cc:205] Tried to move back new op lower bound from 7023240864355160064 to 7023240863350599680. Current Snapshot: MvccSnapshot[applied={T|T < 7023240864355160064}]
INSERT report
rows total: 2000
time total: 131.511 ms
time per row: 0.0657557 ms
I20240502 14:00:14.229727 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:14.235690 522 tablet_service.cc:1467] Processing CreateTablet for tablet be4331e57df84e4ea7f92ff547cb6171 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.235852 387 tablet_service.cc:1467] Processing CreateTablet for tablet be4331e57df84e4ea7f92ff547cb6171 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.235864 386 tablet_service.cc:1467] Processing CreateTablet for tablet 457409778e844247a63aab5187494c74 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.236092 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4331e57df84e4ea7f92ff547cb6171. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.236135 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4331e57df84e4ea7f92ff547cb6171. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.236249 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 457409778e844247a63aab5187494c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.238349 388 tablet_service.cc:1467] Processing CreateTablet for tablet cc005fc5439b4ee6ac9beae8685a842d (DEFAULT_TABLE table=rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.238584 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc005fc5439b4ee6ac9beae8685a842d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.238955 784 tablet_service.cc:1467] Processing CreateTablet for tablet cc005fc5439b4ee6ac9beae8685a842d (DEFAULT_TABLE table=rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.239228 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc005fc5439b4ee6ac9beae8685a842d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.239358 654 tablet_service.cc:1467] Processing CreateTablet for tablet be4331e57df84e4ea7f92ff547cb6171 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.239615 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4331e57df84e4ea7f92ff547cb6171. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.242113 1968 tablet_bootstrap.cc:492] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:14.242344 1966 tablet_bootstrap.cc:492] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:14.243115 1968 tablet_bootstrap.cc:654] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.243556 1966 tablet_bootstrap.cc:654] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.244131 1967 tablet_bootstrap.cc:492] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:14.244968 1967 tablet_bootstrap.cc:654] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.244973 1966 tablet_bootstrap.cc:492] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:14.245060 1966 ts_tablet_manager.cc:1367] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:14.245672 1966 raft_consensus.cc:348] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:14.245800 1966 raft_consensus.cc:374] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.245839 1966 raft_consensus.cc:729] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.245998 1966 consensus_queue.cc:260] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:14.246248 1967 tablet_bootstrap.cc:492] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:14.246295 1966 ts_tablet_manager.cc:1398] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:14.246323 1967 ts_tablet_manager.cc:1367] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:14.246490 1966 tablet_bootstrap.cc:492] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:14.246760 1967 raft_consensus.cc:348] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:14.246870 1967 raft_consensus.cc:374] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.246897 1967 raft_consensus.cc:729] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.247006 1967 consensus_queue.cc:260] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:14.247269 1967 ts_tablet_manager.cc:1398] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.247419 1966 tablet_bootstrap.cc:654] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.250347 654 tablet_service.cc:1467] Processing CreateTablet for tablet 457409778e844247a63aab5187494c74 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.250638 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 457409778e844247a63aab5187494c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.251696 522 tablet_service.cc:1467] Processing CreateTablet for tablet cc005fc5439b4ee6ac9beae8685a842d (DEFAULT_TABLE table=rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.252012 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc005fc5439b4ee6ac9beae8685a842d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.252570 1968 tablet_bootstrap.cc:492] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:14.252640 1968 ts_tablet_manager.cc:1367] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:00:14.253000 1968 raft_consensus.cc:348] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:14.253080 1968 raft_consensus.cc:374] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.253103 1968 raft_consensus.cc:729] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.253101 785 tablet_service.cc:1467] Processing CreateTablet for tablet 457409778e844247a63aab5187494c74 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.253196 1968 consensus_queue.cc:260] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:14.253366 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 457409778e844247a63aab5187494c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.253448 1968 ts_tablet_manager.cc:1398] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:14.253664 1968 tablet_bootstrap.cc:492] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:14.253685 1972 tablet_bootstrap.cc:492] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:14.254055 1967 tablet_bootstrap.cc:492] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:14.254562 1972 tablet_bootstrap.cc:654] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.254562 1968 tablet_bootstrap.cc:654] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.255024 1967 tablet_bootstrap.cc:654] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.258023 1968 tablet_bootstrap.cc:492] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:14.258039 1966 tablet_bootstrap.cc:492] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:14.258121 1968 ts_tablet_manager.cc:1367] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:14.258121 1966 ts_tablet_manager.cc:1367] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240502 14:00:14.258554 1972 tablet_bootstrap.cc:492] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:14.258564 1967 tablet_bootstrap.cc:492] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:14.258646 1972 ts_tablet_manager.cc:1367] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:14.258646 1967 ts_tablet_manager.cc:1367] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:14.258626 1966 raft_consensus.cc:348] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.258738 1966 raft_consensus.cc:374] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.258778 1966 raft_consensus.cc:729] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.258910 1966 consensus_queue.cc:260] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.259104 1967 raft_consensus.cc:348] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.259101 1972 raft_consensus.cc:348] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.259181 1966 ts_tablet_manager.cc:1398] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.259194 1967 raft_consensus.cc:374] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.259212 1972 raft_consensus.cc:374] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.259227 1967 raft_consensus.cc:729] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.259254 1972 raft_consensus.cc:729] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.259352 1966 tablet_bootstrap.cc:492] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:14.259356 1967 consensus_queue.cc:260] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.259377 1972 consensus_queue.cc:260] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.259626 1967 ts_tablet_manager.cc:1398] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.259665 1972 ts_tablet_manager.cc:1398] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.259850 1972 tablet_bootstrap.cc:492] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:14.260375 1966 tablet_bootstrap.cc:654] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.260769 1972 tablet_bootstrap.cc:654] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.260859 1968 raft_consensus.cc:348] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.260998 1968 raft_consensus.cc:374] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.261042 1968 raft_consensus.cc:729] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.261188 1968 consensus_queue.cc:260] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.261586 1968 ts_tablet_manager.cc:1398] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.262590 1966 tablet_bootstrap.cc:492] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:14.262686 1966 ts_tablet_manager.cc:1367] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:14.263137 1966 raft_consensus.cc:348] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.263247 1966 raft_consensus.cc:374] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.263288 1966 raft_consensus.cc:729] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.263411 1966 consensus_queue.cc:260] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.263666 1966 ts_tablet_manager.cc:1398] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.264542 1972 tablet_bootstrap.cc:492] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:14.264639 1972 ts_tablet_manager.cc:1367] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:14.265048 1972 raft_consensus.cc:348] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.265134 1972 raft_consensus.cc:374] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.265169 1972 raft_consensus.cc:729] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.265308 1972 consensus_queue.cc:260] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.265581 1972 ts_tablet_manager.cc:1398] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.275918 1862 raft_consensus.cc:482] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:14.276021 1862 raft_consensus.cc:504] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.276435 1862 leader_election.cc:288] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:14.276760 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc005fc5439b4ee6ac9beae8685a842d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:14.276813 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc005fc5439b4ee6ac9beae8685a842d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:14.276901 409 raft_consensus.cc:2454] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:14.276983 542 raft_consensus.cc:2454] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:14.277168 740 leader_election.cc:302] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:14.277334 1862 raft_consensus.cc:2781] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:14.277401 1862 raft_consensus.cc:482] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:14.277436 1862 raft_consensus.cc:3037] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.278245 1862 raft_consensus.cc:504] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.278564 1862 leader_election.cc:288] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:14.278869 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc005fc5439b4ee6ac9beae8685a842d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:14.278892 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc005fc5439b4ee6ac9beae8685a842d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:14.278990 409 raft_consensus.cc:3037] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.278995 542 raft_consensus.cc:3037] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.279691 542 raft_consensus.cc:2454] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:14.279877 409 raft_consensus.cc:2454] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:14.279989 738 leader_election.cc:302] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:14.280148 1862 raft_consensus.cc:2781] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:14.280308 1862 raft_consensus.cc:686] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:14.280443 1862 consensus_queue.cc:237] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.282099 32734 catalog_manager.cc:5484] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:14.382931 1939 raft_consensus.cc:482] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:14.383066 1939 raft_consensus.cc:504] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.383489 1939 leader_election.cc:288] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:14.383920 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "457409778e844247a63aab5187494c74" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:14.383930 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "457409778e844247a63aab5187494c74" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:14.384056 409 raft_consensus.cc:2454] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:14.384065 806 raft_consensus.cc:2454] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:14.384338 608 leader_election.cc:302] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:14.384488 1939 raft_consensus.cc:2781] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:14.384559 1939 raft_consensus.cc:482] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:14.384609 1939 raft_consensus.cc:3037] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.385461 1939 raft_consensus.cc:504] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.385778 1939 leader_election.cc:288] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:14.386101 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "457409778e844247a63aab5187494c74" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:14.386133 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "457409778e844247a63aab5187494c74" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:14.386224 806 raft_consensus.cc:3037] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.386246 409 raft_consensus.cc:3037] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.386986 806 raft_consensus.cc:2454] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:14.387023 409 raft_consensus.cc:2454] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:14.387275 609 leader_election.cc:302] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:14.387501 1939 raft_consensus.cc:2781] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:14.387574 1939 raft_consensus.cc:686] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:14.387741 1939 consensus_queue.cc:237] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.389513 32734 catalog_manager.cc:5484] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:14.604707 1751 raft_consensus.cc:482] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:14.604835 1751 raft_consensus.cc:504] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:14.605204 1751 leader_election.cc:288] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:14.605671 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4331e57df84e4ea7f92ff547cb6171" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:14.605696 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4331e57df84e4ea7f92ff547cb6171" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:14.605842 542 raft_consensus.cc:2454] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:14.605842 672 raft_consensus.cc:2454] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:14.606112 341 leader_election.cc:302] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:14.606300 1751 raft_consensus.cc:2781] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:14.606356 1751 raft_consensus.cc:482] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:14.606400 1751 raft_consensus.cc:3037] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.607378 1751 raft_consensus.cc:504] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:14.607666 1751 leader_election.cc:288] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:14.608017 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4331e57df84e4ea7f92ff547cb6171" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:14.608050 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4331e57df84e4ea7f92ff547cb6171" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:14.608150 672 raft_consensus.cc:3037] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.608147 542 raft_consensus.cc:3037] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.609048 672 raft_consensus.cc:2454] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:14.609099 542 raft_consensus.cc:2454] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:14.609294 341 leader_election.cc:302] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:14.609459 1751 raft_consensus.cc:2781] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:14.609653 1751 raft_consensus.cc:686] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:14.609797 1751 consensus_queue.cc:237] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:14.612072 32734 catalog_manager.cc:5484] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:14.727675 1985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:14.727974 1985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:14.731267 1985 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240502 14:00:14.808830 542 raft_consensus.cc:1261] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:14.809186 409 raft_consensus.cc:1261] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:14.809792 1862 consensus_queue.cc:1035] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:14.810153 1862 consensus_queue.cc:1035] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:14.810703 672 raft_consensus.cc:1261] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:14.811117 1751 consensus_queue.cc:1035] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:14.816985 408 raft_consensus.cc:1261] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:14.817670 1939 consensus_queue.cc:1035] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:14.818239 806 raft_consensus.cc:1261] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:14.819864 1939 consensus_queue.cc:1035] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:14.825846 542 raft_consensus.cc:1261] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:14.826419 1983 consensus_queue.cc:1035] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 107.485 ms
time per row: 0.0537425 ms
I20240502 14:00:14.883026 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:14.888317 522 tablet_service.cc:1467] Processing CreateTablet for tablet 8568aa57fce64ccf89a36deb5b7b0f3b (DEFAULT_TABLE table=rebalance_test_table_9 [id=ff4d6b4c29c045f2b6b5bc172c5fd4d3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.888434 654 tablet_service.cc:1467] Processing CreateTablet for tablet 8568aa57fce64ccf89a36deb5b7b0f3b (DEFAULT_TABLE table=rebalance_test_table_9 [id=ff4d6b4c29c045f2b6b5bc172c5fd4d3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.888634 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8568aa57fce64ccf89a36deb5b7b0f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.888737 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8568aa57fce64ccf89a36deb5b7b0f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.889359 785 tablet_service.cc:1467] Processing CreateTablet for tablet 8568aa57fce64ccf89a36deb5b7b0f3b (DEFAULT_TABLE table=rebalance_test_table_9 [id=ff4d6b4c29c045f2b6b5bc172c5fd4d3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.889392 520 tablet_service.cc:1467] Processing CreateTablet for tablet 0f076d48336a4828b9b291cc9318e3bc (DEFAULT_TABLE table=rebalance_test_table_9 [id=ff4d6b4c29c045f2b6b5bc172c5fd4d3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.889628 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f076d48336a4828b9b291cc9318e3bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.889628 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8568aa57fce64ccf89a36deb5b7b0f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.891088 784 tablet_service.cc:1467] Processing CreateTablet for tablet 7e9f86a6149d43f3a217769865d9a3d9 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ff4d6b4c29c045f2b6b5bc172c5fd4d3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.891140 521 tablet_service.cc:1467] Processing CreateTablet for tablet 7e9f86a6149d43f3a217769865d9a3d9 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ff4d6b4c29c045f2b6b5bc172c5fd4d3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.891328 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e9f86a6149d43f3a217769865d9a3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.891357 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e9f86a6149d43f3a217769865d9a3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.891840 2053 tablet_bootstrap.cc:492] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:14.892176 2052 tablet_bootstrap.cc:492] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:14.892761 2053 tablet_bootstrap.cc:654] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.893086 2052 tablet_bootstrap.cc:654] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.893755 653 tablet_service.cc:1467] Processing CreateTablet for tablet 0f076d48336a4828b9b291cc9318e3bc (DEFAULT_TABLE table=rebalance_test_table_9 [id=ff4d6b4c29c045f2b6b5bc172c5fd4d3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.894006 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f076d48336a4828b9b291cc9318e3bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.894028 388 tablet_service.cc:1467] Processing CreateTablet for tablet 0f076d48336a4828b9b291cc9318e3bc (DEFAULT_TABLE table=rebalance_test_table_9 [id=ff4d6b4c29c045f2b6b5bc172c5fd4d3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.894290 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f076d48336a4828b9b291cc9318e3bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.894299 2052 tablet_bootstrap.cc:492] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:14.894371 2052 ts_tablet_manager.cc:1367] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:14.894814 2052 raft_consensus.cc:348] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.894948 2052 raft_consensus.cc:374] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.894987 2052 raft_consensus.cc:729] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.895109 2052 consensus_queue.cc:260] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.895368 2052 ts_tablet_manager.cc:1398] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.896270 654 tablet_service.cc:1467] Processing CreateTablet for tablet 7e9f86a6149d43f3a217769865d9a3d9 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ff4d6b4c29c045f2b6b5bc172c5fd4d3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:14.896486 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e9f86a6149d43f3a217769865d9a3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:14.896718 2052 tablet_bootstrap.cc:492] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:14.896852 2056 tablet_bootstrap.cc:492] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:14.897501 2056 tablet_bootstrap.cc:654] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.897563 2052 tablet_bootstrap.cc:654] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.898437 2053 tablet_bootstrap.cc:492] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:14.898437 2056 tablet_bootstrap.cc:492] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:14.898519 2053 ts_tablet_manager.cc:1367] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:14.898514 2056 ts_tablet_manager.cc:1367] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:14.898630 2051 tablet_bootstrap.cc:492] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:14.898952 2056 raft_consensus.cc:348] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.898991 2053 raft_consensus.cc:348] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.899082 2053 raft_consensus.cc:374] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.899083 2056 raft_consensus.cc:374] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.899125 2056 raft_consensus.cc:729] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.899158 2053 raft_consensus.cc:729] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.899298 2056 consensus_queue.cc:260] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.899339 2053 consensus_queue.cc:260] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.899749 2051 tablet_bootstrap.cc:654] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.899781 2053 ts_tablet_manager.cc:1398] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.900449 2052 tablet_bootstrap.cc:492] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:14.900542 2052 ts_tablet_manager.cc:1367] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:14.900655 2056 ts_tablet_manager.cc:1398] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:14.900923 2052 raft_consensus.cc:348] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.901018 2052 raft_consensus.cc:374] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.901049 2052 raft_consensus.cc:729] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.901183 2053 tablet_bootstrap.cc:492] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:14.901154 2052 consensus_queue.cc:260] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.901397 2052 ts_tablet_manager.cc:1398] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.901572 2052 tablet_bootstrap.cc:492] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:14.902047 2053 tablet_bootstrap.cc:654] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.902659 2052 tablet_bootstrap.cc:654] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.903353 2053 tablet_bootstrap.cc:492] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:14.903440 2053 ts_tablet_manager.cc:1367] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:14.903785 2053 raft_consensus.cc:348] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.903931 2053 raft_consensus.cc:374] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.903971 2053 raft_consensus.cc:729] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.904129 2053 consensus_queue.cc:260] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.904456 2053 ts_tablet_manager.cc:1398] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.904929 2052 tablet_bootstrap.cc:492] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:14.905020 2052 ts_tablet_manager.cc:1367] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:14.905450 2052 raft_consensus.cc:348] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.905589 2052 raft_consensus.cc:374] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.905637 2052 raft_consensus.cc:729] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.905786 2052 consensus_queue.cc:260] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.906064 2051 tablet_bootstrap.cc:492] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:14.906150 2051 ts_tablet_manager.cc:1367] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:14.906512 2052 ts_tablet_manager.cc:1398] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.906599 2051 raft_consensus.cc:348] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.906718 2051 raft_consensus.cc:374] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.906765 2051 raft_consensus.cc:729] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.906898 2051 consensus_queue.cc:260] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.907243 2051 ts_tablet_manager.cc:1398] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.907406 2051 tablet_bootstrap.cc:492] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:14.908725 2051 tablet_bootstrap.cc:654] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.909901 2051 tablet_bootstrap.cc:492] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:14.909998 2051 ts_tablet_manager.cc:1367] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:14.910368 2051 raft_consensus.cc:348] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.910455 2051 raft_consensus.cc:374] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.910487 2051 raft_consensus.cc:729] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.910588 2051 consensus_queue.cc:260] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.910835 2051 ts_tablet_manager.cc:1398] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:14.911057 2051 tablet_bootstrap.cc:492] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:14.911979 2051 tablet_bootstrap.cc:654] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:14.913046 2051 tablet_bootstrap.cc:492] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:14.913126 2051 ts_tablet_manager.cc:1367] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:14.913463 2051 raft_consensus.cc:348] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.913575 2051 raft_consensus.cc:374] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:14.913626 2051 raft_consensus.cc:729] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:14.913839 2051 consensus_queue.cc:260] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.914080 2051 ts_tablet_manager.cc:1398] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:14.916919 1962 raft_consensus.cc:482] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:14.916998 1962 raft_consensus.cc:504] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.917358 1962 leader_election.cc:288] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:14.917670 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8568aa57fce64ccf89a36deb5b7b0f3b" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:14.917716 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8568aa57fce64ccf89a36deb5b7b0f3b" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:14.917791 672 raft_consensus.cc:2454] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:14.917842 806 raft_consensus.cc:2454] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:14.918087 474 leader_election.cc:302] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:14.918229 1962 raft_consensus.cc:2781] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:14.918290 1962 raft_consensus.cc:482] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:14.918329 1962 raft_consensus.cc:3037] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.919101 1962 raft_consensus.cc:504] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.919402 1962 leader_election.cc:288] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:14.919723 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8568aa57fce64ccf89a36deb5b7b0f3b" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:14.919720 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8568aa57fce64ccf89a36deb5b7b0f3b" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:14.919867 672 raft_consensus.cc:3037] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.919869 806 raft_consensus.cc:3037] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.920827 672 raft_consensus.cc:2454] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:14.920827 806 raft_consensus.cc:2454] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:14.921113 474 leader_election.cc:302] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:14.921281 1962 raft_consensus.cc:2781] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:14.921376 1962 raft_consensus.cc:686] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:14.921512 1962 consensus_queue.cc:237] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:14.923148 32729 catalog_manager.cc:5484] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:14.953189 2044 raft_consensus.cc:482] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:14.953316 2044 raft_consensus.cc:504] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.953756 2044 leader_election.cc:288] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:14.954145 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e9f86a6149d43f3a217769865d9a3d9" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:14.954149 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e9f86a6149d43f3a217769865d9a3d9" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:14.954275 805 raft_consensus.cc:2454] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:14.954281 542 raft_consensus.cc:2454] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:14.954582 606 leader_election.cc:302] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:14.954739 2044 raft_consensus.cc:2781] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:14.954815 2044 raft_consensus.cc:482] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:14.954869 2044 raft_consensus.cc:3037] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.955869 2044 raft_consensus.cc:504] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.956187 2044 leader_election.cc:288] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:14.956508 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e9f86a6149d43f3a217769865d9a3d9" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:14.956557 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e9f86a6149d43f3a217769865d9a3d9" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:14.956635 805 raft_consensus.cc:3037] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.956663 542 raft_consensus.cc:3037] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:14.957566 805 raft_consensus.cc:2454] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:14.957566 542 raft_consensus.cc:2454] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:14.957834 609 leader_election.cc:302] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:14.958000 2044 raft_consensus.cc:2781] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:14.958086 2044 raft_consensus.cc:686] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:14.958237 2044 consensus_queue.cc:237] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:14.960083 32729 catalog_manager.cc:5484] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:15.235942 2044 raft_consensus.cc:482] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:15.236086 2044 raft_consensus.cc:504] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.236497 2044 leader_election.cc:288] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:15.236886 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f076d48336a4828b9b291cc9318e3bc" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:15.236917 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f076d48336a4828b9b291cc9318e3bc" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:15.237031 542 raft_consensus.cc:2454] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:15.237049 408 raft_consensus.cc:2454] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:15.237332 606 leader_election.cc:302] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:15.237572 2044 raft_consensus.cc:2781] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:15.237622 2044 raft_consensus.cc:482] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:15.237665 2044 raft_consensus.cc:3037] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.238692 2044 raft_consensus.cc:504] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.239014 2044 leader_election.cc:288] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:15.239295 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f076d48336a4828b9b291cc9318e3bc" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:15.239320 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f076d48336a4828b9b291cc9318e3bc" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:15.239408 408 raft_consensus.cc:3037] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.239423 542 raft_consensus.cc:3037] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.240350 408 raft_consensus.cc:2454] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:15.240428 542 raft_consensus.cc:2454] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:15.240676 608 leader_election.cc:302] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:15.240824 2044 raft_consensus.cc:2781] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:15.240931 2044 raft_consensus.cc:686] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:15.241063 2044 consensus_queue.cc:237] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.243980 32729 catalog_manager.cc:5484] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:15.330500 1962 consensus_queue.cc:1035] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:15.336115 1963 consensus_queue.cc:1035] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 14:00:15.378813 2067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:15.379055 2067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:15.382323 2067 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 14:00:15.393590 2044 consensus_queue.cc:1035] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:15.399576 1939 consensus_queue.cc:1035] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_9'
I20240502 14:00:15.490002 542 raft_consensus.cc:1261] T 0f076d48336a4828b9b291cc9318e3bc P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:15.490190 408 raft_consensus.cc:1261] T 0f076d48336a4828b9b291cc9318e3bc P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:15.490581 1939 consensus_queue.cc:1035] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:15.494750 2044 consensus_queue.cc:1035] T 0f076d48336a4828b9b291cc9318e3bc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 118.88 ms
time per row: 0.0594402 ms
I20240502 14:00:15.557061 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:15.562739 784 tablet_service.cc:1467] Processing CreateTablet for tablet 8f862cbcf8fc49598ef9cfbca658fc5b (DEFAULT_TABLE table=rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:15.562786 388 tablet_service.cc:1467] Processing CreateTablet for tablet 8f862cbcf8fc49598ef9cfbca658fc5b (DEFAULT_TABLE table=rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:15.563105 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f862cbcf8fc49598ef9cfbca658fc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:15.563119 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f862cbcf8fc49598ef9cfbca658fc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:15.563702 521 tablet_service.cc:1467] Processing CreateTablet for tablet 8f862cbcf8fc49598ef9cfbca658fc5b (DEFAULT_TABLE table=rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:15.563998 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f862cbcf8fc49598ef9cfbca658fc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:15.564394 654 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebc7252d1534e31b23b79c806416bee (DEFAULT_TABLE table=rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:15.564656 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebc7252d1534e31b23b79c806416bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:15.565511 2126 tablet_bootstrap.cc:492] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:15.565912 387 tablet_service.cc:1467] Processing CreateTablet for tablet efe3db917ec341dc935709b8e66e969b (DEFAULT_TABLE table=rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:15.566151 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efe3db917ec341dc935709b8e66e969b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:15.566421 2126 tablet_bootstrap.cc:654] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:15.566406 520 tablet_service.cc:1467] Processing CreateTablet for tablet efe3db917ec341dc935709b8e66e969b (DEFAULT_TABLE table=rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:15.566627 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efe3db917ec341dc935709b8e66e969b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:15.567165 2128 tablet_bootstrap.cc:492] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:15.568512 2128 tablet_bootstrap.cc:654] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:15.568596 2129 tablet_bootstrap.cc:492] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:15.568629 388 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebc7252d1534e31b23b79c806416bee (DEFAULT_TABLE table=rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:15.568843 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebc7252d1534e31b23b79c806416bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:15.569442 2129 tablet_bootstrap.cc:654] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:15.569526 784 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebc7252d1534e31b23b79c806416bee (DEFAULT_TABLE table=rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:15.569780 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebc7252d1534e31b23b79c806416bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:15.570354 785 tablet_service.cc:1467] Processing CreateTablet for tablet efe3db917ec341dc935709b8e66e969b (DEFAULT_TABLE table=rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:15.570416 2126 tablet_bootstrap.cc:492] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:15.570537 2126 ts_tablet_manager.cc:1367] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:15.570597 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efe3db917ec341dc935709b8e66e969b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:15.570989 2126 raft_consensus.cc:348] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.571085 2126 raft_consensus.cc:374] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:15.571113 2126 raft_consensus.cc:729] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:15.571225 2126 consensus_queue.cc:260] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.571506 2126 ts_tablet_manager.cc:1398] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:15.571743 2127 tablet_bootstrap.cc:492] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:15.572072 2129 tablet_bootstrap.cc:492] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:15.572152 2129 ts_tablet_manager.cc:1367] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:15.572397 2126 tablet_bootstrap.cc:492] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:15.572763 2129 raft_consensus.cc:348] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:15.572984 2129 raft_consensus.cc:374] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:15.573081 2129 raft_consensus.cc:729] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:15.573287 2126 tablet_bootstrap.cc:654] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:15.573282 2129 consensus_queue.cc:260] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:15.573583 2129 ts_tablet_manager.cc:1398] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:15.574517 2127 tablet_bootstrap.cc:654] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:15.574709 2126 tablet_bootstrap.cc:492] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:15.574820 2126 ts_tablet_manager.cc:1367] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:15.575083 2128 tablet_bootstrap.cc:492] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:15.575181 2128 ts_tablet_manager.cc:1367] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:15.575258 2126 raft_consensus.cc:348] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:15.575376 2126 raft_consensus.cc:374] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:15.575412 2126 raft_consensus.cc:729] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:15.575539 2126 consensus_queue.cc:260] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:15.575695 2128 raft_consensus.cc:348] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.575793 2128 raft_consensus.cc:374] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:15.575851 2128 raft_consensus.cc:729] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:15.575927 2126 ts_tablet_manager.cc:1398] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:15.576004 2127 tablet_bootstrap.cc:492] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:15.575999 2128 consensus_queue.cc:260] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.576086 2127 ts_tablet_manager.cc:1367] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:15.576157 2126 tablet_bootstrap.cc:492] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:15.576267 2128 ts_tablet_manager.cc:1398] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:15.576417 2128 tablet_bootstrap.cc:492] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:15.576772 2127 raft_consensus.cc:348] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.576961 2127 raft_consensus.cc:374] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:15.577055 2126 tablet_bootstrap.cc:654] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:15.577068 2127 raft_consensus.cc:729] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:15.577390 2128 tablet_bootstrap.cc:654] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:15.577428 2127 consensus_queue.cc:260] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.577993 2127 ts_tablet_manager.cc:1398] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:15.578176 2127 tablet_bootstrap.cc:492] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:15.579037 2127 tablet_bootstrap.cc:654] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:15.580109 2127 tablet_bootstrap.cc:492] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:15.580192 2127 ts_tablet_manager.cc:1367] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:15.580521 2127 raft_consensus.cc:348] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.580606 2127 raft_consensus.cc:374] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:15.580638 2127 raft_consensus.cc:729] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:15.580760 2127 consensus_queue.cc:260] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.581043 2127 ts_tablet_manager.cc:1398] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:15.581213 2127 tablet_bootstrap.cc:492] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:15.581219 2128 tablet_bootstrap.cc:492] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:15.581308 2128 ts_tablet_manager.cc:1367] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:15.581678 2128 raft_consensus.cc:348] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.581797 2128 raft_consensus.cc:374] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:15.581848 2128 raft_consensus.cc:729] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:15.581976 2128 consensus_queue.cc:260] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.582113 2127 tablet_bootstrap.cc:654] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:15.582262 2128 ts_tablet_manager.cc:1398] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:15.583308 2127 tablet_bootstrap.cc:492] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:15.583400 2127 ts_tablet_manager.cc:1367] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:15.583593 2126 tablet_bootstrap.cc:492] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:15.583678 2126 ts_tablet_manager.cc:1367] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 14:00:15.583856 2127 raft_consensus.cc:348] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:15.583982 2127 raft_consensus.cc:374] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:15.584021 2127 raft_consensus.cc:729] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:15.584126 2126 raft_consensus.cc:348] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.584159 2127 consensus_queue.cc:260] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:15.584239 2126 raft_consensus.cc:374] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:15.584370 2126 raft_consensus.cc:729] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:15.584434 2127 ts_tablet_manager.cc:1398] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:15.584537 2126 consensus_queue.cc:260] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.584918 2126 ts_tablet_manager.cc:1398] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:15.628280 2044 raft_consensus.cc:482] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:15.628398 2044 raft_consensus.cc:504] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:15.628728 2044 leader_election.cc:288] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:15.629076 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebc7252d1534e31b23b79c806416bee" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:15.629094 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebc7252d1534e31b23b79c806416bee" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:15.629213 806 raft_consensus.cc:2454] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:15.629240 408 raft_consensus.cc:2454] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:15.629508 609 leader_election.cc:302] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:15.629675 2044 raft_consensus.cc:2781] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:15.629731 2044 raft_consensus.cc:482] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:15.629783 2044 raft_consensus.cc:3037] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.630628 2044 raft_consensus.cc:504] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:15.630939 2044 leader_election.cc:288] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:15.631269 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebc7252d1534e31b23b79c806416bee" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:15.631287 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebc7252d1534e31b23b79c806416bee" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:15.631374 806 raft_consensus.cc:3037] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.631382 408 raft_consensus.cc:3037] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.632176 408 raft_consensus.cc:2454] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:15.632180 806 raft_consensus.cc:2454] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:15.632467 609 leader_election.cc:302] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:15.632630 2044 raft_consensus.cc:2781] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:15.632716 2044 raft_consensus.cc:686] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:15.632849 2044 consensus_queue.cc:237] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:15.634716 32729 catalog_manager.cc:5484] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:15.677757 2034 raft_consensus.cc:482] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:15.677904 2034 raft_consensus.cc:504] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.678324 2034 leader_election.cc:288] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:15.678753 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe3db917ec341dc935709b8e66e969b" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:15.678776 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe3db917ec341dc935709b8e66e969b" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:15.678915 541 raft_consensus.cc:2454] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:15.678951 408 raft_consensus.cc:2454] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:15.679227 738 leader_election.cc:302] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:15.679374 2034 raft_consensus.cc:2781] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:15.679435 2034 raft_consensus.cc:482] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:15.679489 2034 raft_consensus.cc:3037] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.680712 2034 raft_consensus.cc:504] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.681071 2034 leader_election.cc:288] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:15.681423 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe3db917ec341dc935709b8e66e969b" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:15.681425 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe3db917ec341dc935709b8e66e969b" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:15.681535 408 raft_consensus.cc:3037] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.681545 541 raft_consensus.cc:3037] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.682363 541 raft_consensus.cc:2454] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:15.682382 408 raft_consensus.cc:2454] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:15.682654 738 leader_election.cc:302] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:15.682832 2034 raft_consensus.cc:2781] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:15.682909 2034 raft_consensus.cc:686] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:15.683064 2034 consensus_queue.cc:237] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.685005 32729 catalog_manager.cc:5484] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:15.885812 2075 raft_consensus.cc:482] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:15.885957 2075 raft_consensus.cc:504] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.886438 2075 leader_election.cc:288] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:15.886871 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f862cbcf8fc49598ef9cfbca658fc5b" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:15.886898 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f862cbcf8fc49598ef9cfbca658fc5b" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:15.887012 806 raft_consensus.cc:2454] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:15.887060 541 raft_consensus.cc:2454] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:15.887312 344 leader_election.cc:302] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:15.887508 2075 raft_consensus.cc:2781] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:15.887567 2075 raft_consensus.cc:482] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:15.887609 2075 raft_consensus.cc:3037] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.888573 2075 raft_consensus.cc:504] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.888882 2075 leader_election.cc:288] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:15.889182 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f862cbcf8fc49598ef9cfbca658fc5b" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:15.889182 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f862cbcf8fc49598ef9cfbca658fc5b" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:15.889309 541 raft_consensus.cc:3037] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.889309 806 raft_consensus.cc:3037] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:15.890244 541 raft_consensus.cc:2454] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:15.890244 806 raft_consensus.cc:2454] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:15.890506 344 leader_election.cc:302] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:15.890659 2141 raft_consensus.cc:2781] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:15.890745 2141 raft_consensus.cc:686] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:15.890898 2141 consensus_queue.cc:237] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:15.892586 32729 catalog_manager.cc:5484] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:16.056094 2143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:16.056382 2143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:16.059532 2143 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 14:00:16.075496 2034 consensus_queue.cc:1035] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:16.079509 2042 consensus_queue.cc:1035] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_10'
I20240502 14:00:16.122143 2118 consensus_queue.cc:1035] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:16.129047 2118 consensus_queue.cc:1035] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:16.168164 804 raft_consensus.cc:1261] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:16.169390 2075 consensus_queue.cc:1035] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:16.170030 541 raft_consensus.cc:1261] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:16.170482 2075 consensus_queue.cc:1035] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 128.526 ms
time per row: 0.0642632 ms
I20240502 14:00:16.234066 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:16.240389 388 tablet_service.cc:1467] Processing CreateTablet for tablet ce81659e36874bb89da4d2268eb40aea (DEFAULT_TABLE table=rebalance_test_table_11 [id=441df53b66fd4ef7bb803ffdac4eb194]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.240506 520 tablet_service.cc:1467] Processing CreateTablet for tablet ce81659e36874bb89da4d2268eb40aea (DEFAULT_TABLE table=rebalance_test_table_11 [id=441df53b66fd4ef7bb803ffdac4eb194]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.240729 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce81659e36874bb89da4d2268eb40aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.240818 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce81659e36874bb89da4d2268eb40aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.245026 653 tablet_service.cc:1467] Processing CreateTablet for tablet d37a7cd2db2a431caaba0257f9534e3b (DEFAULT_TABLE table=rebalance_test_table_11 [id=441df53b66fd4ef7bb803ffdac4eb194]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.245355 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37a7cd2db2a431caaba0257f9534e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.245303 521 tablet_service.cc:1467] Processing CreateTablet for tablet 8689975c1d9d41f4ae7152bf6e4a58a8 (DEFAULT_TABLE table=rebalance_test_table_11 [id=441df53b66fd4ef7bb803ffdac4eb194]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.245558 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8689975c1d9d41f4ae7152bf6e4a58a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.245782 388 tablet_service.cc:1467] Processing CreateTablet for tablet d37a7cd2db2a431caaba0257f9534e3b (DEFAULT_TABLE table=rebalance_test_table_11 [id=441df53b66fd4ef7bb803ffdac4eb194]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.246058 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37a7cd2db2a431caaba0257f9534e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.245026 654 tablet_service.cc:1467] Processing CreateTablet for tablet ce81659e36874bb89da4d2268eb40aea (DEFAULT_TABLE table=rebalance_test_table_11 [id=441df53b66fd4ef7bb803ffdac4eb194]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.247916 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce81659e36874bb89da4d2268eb40aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.248247 2203 tablet_bootstrap.cc:492] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:16.248733 785 tablet_service.cc:1467] Processing CreateTablet for tablet d37a7cd2db2a431caaba0257f9534e3b (DEFAULT_TABLE table=rebalance_test_table_11 [id=441df53b66fd4ef7bb803ffdac4eb194]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.248787 784 tablet_service.cc:1467] Processing CreateTablet for tablet 8689975c1d9d41f4ae7152bf6e4a58a8 (DEFAULT_TABLE table=rebalance_test_table_11 [id=441df53b66fd4ef7bb803ffdac4eb194]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.248996 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37a7cd2db2a431caaba0257f9534e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.249158 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8689975c1d9d41f4ae7152bf6e4a58a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.250109 653 tablet_service.cc:1467] Processing CreateTablet for tablet 8689975c1d9d41f4ae7152bf6e4a58a8 (DEFAULT_TABLE table=rebalance_test_table_11 [id=441df53b66fd4ef7bb803ffdac4eb194]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.250355 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8689975c1d9d41f4ae7152bf6e4a58a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.252138 2205 tablet_bootstrap.cc:492] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:16.252946 2206 tablet_bootstrap.cc:492] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:16.253453 2203 tablet_bootstrap.cc:654] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.253688 2206 tablet_bootstrap.cc:654] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.254781 2205 tablet_bootstrap.cc:654] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.255002 2203 tablet_bootstrap.cc:492] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:16.255115 2203 ts_tablet_manager.cc:1367] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:16.255723 2203 raft_consensus.cc:348] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:16.255882 2203 raft_consensus.cc:374] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.255919 2203 raft_consensus.cc:729] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.256043 2203 consensus_queue.cc:260] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:16.256350 2203 ts_tablet_manager.cc:1398] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:16.256471 2205 tablet_bootstrap.cc:492] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:16.256558 2205 ts_tablet_manager.cc:1367] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:16.256568 2203 tablet_bootstrap.cc:492] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:16.257004 2205 raft_consensus.cc:348] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:16.257108 2205 raft_consensus.cc:374] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.257138 2205 raft_consensus.cc:729] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.257485 2203 tablet_bootstrap.cc:654] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.257658 2206 tablet_bootstrap.cc:492] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:16.257738 2206 ts_tablet_manager.cc:1367] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:16.258211 2206 raft_consensus.cc:348] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:16.258323 2206 raft_consensus.cc:374] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.258354 2206 raft_consensus.cc:729] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.258476 2206 consensus_queue.cc:260] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:16.258656 2205 consensus_queue.cc:260] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:16.258810 2204 tablet_bootstrap.cc:492] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:16.259034 2205 ts_tablet_manager.cc:1398] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:16.259761 2206 ts_tablet_manager.cc:1398] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:16.259860 2204 tablet_bootstrap.cc:654] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.259974 2206 tablet_bootstrap.cc:492] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:16.260656 2205 tablet_bootstrap.cc:492] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:16.260947 2203 tablet_bootstrap.cc:492] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:16.261032 2203 ts_tablet_manager.cc:1367] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:16.261586 2205 tablet_bootstrap.cc:654] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.261493 2203 raft_consensus.cc:348] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.261659 2203 raft_consensus.cc:374] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.261694 2203 raft_consensus.cc:729] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.261858 2203 consensus_queue.cc:260] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.262271 2203 ts_tablet_manager.cc:1398] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:16.262619 2204 tablet_bootstrap.cc:492] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:16.262738 2204 ts_tablet_manager.cc:1367] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:16.263219 2206 tablet_bootstrap.cc:654] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.263387 2204 raft_consensus.cc:348] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.263530 2204 raft_consensus.cc:374] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.263597 2204 raft_consensus.cc:729] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.263929 2204 consensus_queue.cc:260] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.264775 2206 tablet_bootstrap.cc:492] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:16.264863 2206 ts_tablet_manager.cc:1367] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 14:00:16.265326 2206 raft_consensus.cc:348] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.265446 2206 raft_consensus.cc:374] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.265508 2206 raft_consensus.cc:729] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.265652 2206 consensus_queue.cc:260] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.265901 2204 ts_tablet_manager.cc:1398] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:16.265977 2206 ts_tablet_manager.cc:1398] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 14:00:16.266081 2204 tablet_bootstrap.cc:492] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:16.267521 2205 tablet_bootstrap.cc:492] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:16.267637 2205 ts_tablet_manager.cc:1367] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 14:00:16.267654 2204 tablet_bootstrap.cc:654] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.268194 2205 raft_consensus.cc:348] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:16.268307 2205 raft_consensus.cc:374] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.268340 2205 raft_consensus.cc:729] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.268468 2205 consensus_queue.cc:260] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:16.268635 2204 tablet_bootstrap.cc:492] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:16.268697 2204 ts_tablet_manager.cc:1367] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:16.268797 2205 ts_tablet_manager.cc:1398] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:16.269085 2204 raft_consensus.cc:348] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:16.269186 2204 raft_consensus.cc:374] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.269219 2204 raft_consensus.cc:729] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.269338 2204 consensus_queue.cc:260] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:16.269640 2204 ts_tablet_manager.cc:1398] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:16.269827 2204 tablet_bootstrap.cc:492] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:16.270740 2204 tablet_bootstrap.cc:654] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.271744 2204 tablet_bootstrap.cc:492] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:16.271863 2204 ts_tablet_manager.cc:1367] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:16.272325 2204 raft_consensus.cc:348] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:16.272437 2204 raft_consensus.cc:374] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.272469 2204 raft_consensus.cc:729] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.272606 2204 consensus_queue.cc:260] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:16.272972 2204 ts_tablet_manager.cc:1398] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:16.374663 2075 raft_consensus.cc:482] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:16.374799 2075 raft_consensus.cc:504] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:16.375253 2075 leader_election.cc:288] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:16.375619 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce81659e36874bb89da4d2268eb40aea" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:16.375677 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce81659e36874bb89da4d2268eb40aea" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:16.375758 672 raft_consensus.cc:2454] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:16.375890 541 raft_consensus.cc:2454] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:16.376169 341 leader_election.cc:302] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:16.376335 2075 raft_consensus.cc:2781] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:16.376395 2075 raft_consensus.cc:482] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:16.376435 2075 raft_consensus.cc:3037] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.377303 2075 raft_consensus.cc:504] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:16.377665 2075 leader_election.cc:288] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:16.377992 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce81659e36874bb89da4d2268eb40aea" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:16.378052 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce81659e36874bb89da4d2268eb40aea" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:16.378123 672 raft_consensus.cc:3037] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.378160 541 raft_consensus.cc:3037] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.379118 541 raft_consensus.cc:2454] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:16.379118 672 raft_consensus.cc:2454] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:16.379411 341 leader_election.cc:302] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:16.379570 2075 raft_consensus.cc:2781] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:16.379657 2075 raft_consensus.cc:686] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:16.379808 2075 consensus_queue.cc:237] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:16.381834 32727 catalog_manager.cc:5484] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:16.384644 2183 raft_consensus.cc:482] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:16.384743 2183 raft_consensus.cc:504] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.385059 2183 leader_election.cc:288] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:16.385421 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37a7cd2db2a431caaba0257f9534e3b" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:16.385448 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37a7cd2db2a431caaba0257f9534e3b" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:16.385542 408 raft_consensus.cc:2454] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:16.385589 806 raft_consensus.cc:2454] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:16.385848 608 leader_election.cc:302] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:16.386157 2183 raft_consensus.cc:2781] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:16.386214 2183 raft_consensus.cc:482] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:16.386241 2183 raft_consensus.cc:3037] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.387318 2183 raft_consensus.cc:504] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.387645 2183 leader_election.cc:288] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:16.388007 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37a7cd2db2a431caaba0257f9534e3b" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:16.388007 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37a7cd2db2a431caaba0257f9534e3b" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:16.388134 408 raft_consensus.cc:3037] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.388134 806 raft_consensus.cc:3037] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.389088 408 raft_consensus.cc:2454] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:16.389088 806 raft_consensus.cc:2454] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:16.389386 608 leader_election.cc:302] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:16.389534 2183 raft_consensus.cc:2781] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:16.389626 2183 raft_consensus.cc:686] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:16.389794 2183 consensus_queue.cc:237] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.391662 32727 catalog_manager.cc:5484] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:16.494575 2183 raft_consensus.cc:482] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:16.494707 2183 raft_consensus.cc:504] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:16.495206 2183 leader_election.cc:288] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:16.495577 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8689975c1d9d41f4ae7152bf6e4a58a8" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:16.495587 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8689975c1d9d41f4ae7152bf6e4a58a8" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:16.495734 806 raft_consensus.cc:2454] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:16.495743 541 raft_consensus.cc:2454] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:16.496109 609 leader_election.cc:302] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:16.496295 2183 raft_consensus.cc:2781] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:16.496361 2183 raft_consensus.cc:482] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:16.496412 2183 raft_consensus.cc:3037] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.497267 2183 raft_consensus.cc:504] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:16.497594 2183 leader_election.cc:288] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:16.497888 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8689975c1d9d41f4ae7152bf6e4a58a8" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:16.497922 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8689975c1d9d41f4ae7152bf6e4a58a8" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:16.497995 542 raft_consensus.cc:3037] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.498026 806 raft_consensus.cc:3037] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.498746 542 raft_consensus.cc:2454] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:16.498829 806 raft_consensus.cc:2454] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:16.499037 606 leader_election.cc:302] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:16.499251 2183 raft_consensus.cc:2781] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:16.499343 2183 raft_consensus.cc:686] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:16.499475 2183 consensus_queue.cc:237] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:16.501320 32727 catalog_manager.cc:5484] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:16.664264 2219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:16.664631 2219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:16.668102 2219 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_11'
I20240502 14:00:16.746923 804 raft_consensus.cc:1261] T d37a7cd2db2a431caaba0257f9534e3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:16.746923 408 raft_consensus.cc:1261] T d37a7cd2db2a431caaba0257f9534e3b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:16.747442 2183 consensus_queue.cc:1035] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:16.747706 2044 consensus_queue.cc:1035] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:16.752728 542 raft_consensus.cc:1261] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:16.753191 2183 consensus_queue.cc:1035] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:16.755587 804 raft_consensus.cc:1261] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:16.756414 2183 consensus_queue.cc:1035] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:16.765576 541 raft_consensus.cc:1261] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:16.766186 2075 consensus_queue.cc:1035] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:16.775887 672 raft_consensus.cc:1261] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:16.776557 2141 consensus_queue.cc:1035] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 112.498 ms
time per row: 0.0562489 ms
I20240502 14:00:16.823427 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:16.829221 520 tablet_service.cc:1467] Processing CreateTablet for tablet 95cf29cac53140ac8ab67ad52b95a886 (DEFAULT_TABLE table=rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.829254 388 tablet_service.cc:1467] Processing CreateTablet for tablet 95cf29cac53140ac8ab67ad52b95a886 (DEFAULT_TABLE table=rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.829557 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95cf29cac53140ac8ab67ad52b95a886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.829583 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95cf29cac53140ac8ab67ad52b95a886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.829882 521 tablet_service.cc:1467] Processing CreateTablet for tablet fb75f88d4c7e42f787118144e9f6c430 (DEFAULT_TABLE table=rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.830041 387 tablet_service.cc:1467] Processing CreateTablet for tablet fb75f88d4c7e42f787118144e9f6c430 (DEFAULT_TABLE table=rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.830117 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb75f88d4c7e42f787118144e9f6c430. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.830255 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb75f88d4c7e42f787118144e9f6c430. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.832052 2281 tablet_bootstrap.cc:492] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:16.832115 386 tablet_service.cc:1467] Processing CreateTablet for tablet 34e6f7677fbe40568b51a023be4c1ccd (DEFAULT_TABLE table=rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.832345 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34e6f7677fbe40568b51a023be4c1ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.832531 653 tablet_service.cc:1467] Processing CreateTablet for tablet 95cf29cac53140ac8ab67ad52b95a886 (DEFAULT_TABLE table=rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.832785 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95cf29cac53140ac8ab67ad52b95a886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.833277 2281 tablet_bootstrap.cc:654] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.834383 2281 tablet_bootstrap.cc:492] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:16.834460 2281 ts_tablet_manager.cc:1367] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:16.835013 2284 tablet_bootstrap.cc:492] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:16.835031 2281 raft_consensus.cc:348] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:16.835129 2281 raft_consensus.cc:374] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.835161 2281 raft_consensus.cc:729] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.835119 654 tablet_service.cc:1467] Processing CreateTablet for tablet 34e6f7677fbe40568b51a023be4c1ccd (DEFAULT_TABLE table=rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.835325 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34e6f7677fbe40568b51a023be4c1ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.835309 2281 consensus_queue.cc:260] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:16.835691 2281 ts_tablet_manager.cc:1398] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:16.835891 2281 tablet_bootstrap.cc:492] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:16.836014 2284 tablet_bootstrap.cc:654] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.836768 2281 tablet_bootstrap.cc:654] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.837716 2281 tablet_bootstrap.cc:492] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:16.837716 2284 tablet_bootstrap.cc:492] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:16.837800 2281 ts_tablet_manager.cc:1367] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:16.837800 2284 ts_tablet_manager.cc:1367] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:16.838213 2281 raft_consensus.cc:348] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.838213 2284 raft_consensus.cc:348] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:16.838316 2281 raft_consensus.cc:374] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.838316 2284 raft_consensus.cc:374] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.838349 2281 raft_consensus.cc:729] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.838349 2284 raft_consensus.cc:729] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.838470 2281 consensus_queue.cc:260] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.838773 2281 ts_tablet_manager.cc:1398] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:16.839532 2282 tablet_bootstrap.cc:492] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:16.840360 2284 consensus_queue.cc:260] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:16.840490 785 tablet_service.cc:1467] Processing CreateTablet for tablet fb75f88d4c7e42f787118144e9f6c430 (DEFAULT_TABLE table=rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.840762 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb75f88d4c7e42f787118144e9f6c430. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.840777 2284 ts_tablet_manager.cc:1398] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:00:16.841272 2284 tablet_bootstrap.cc:492] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:16.842154 2284 tablet_bootstrap.cc:654] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.843245 784 tablet_service.cc:1467] Processing CreateTablet for tablet 34e6f7677fbe40568b51a023be4c1ccd (DEFAULT_TABLE table=rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:16.843500 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34e6f7677fbe40568b51a023be4c1ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.843601 2282 tablet_bootstrap.cc:654] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.844703 2284 tablet_bootstrap.cc:492] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:16.844779 2284 ts_tablet_manager.cc:1367] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:16.845394 2284 raft_consensus.cc:348] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.845505 2284 raft_consensus.cc:374] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.845531 2284 raft_consensus.cc:729] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.845628 2284 consensus_queue.cc:260] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.845873 2284 ts_tablet_manager.cc:1398] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:16.847194 2282 tablet_bootstrap.cc:492] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:16.847319 2282 ts_tablet_manager.cc:1367] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:16.847882 2288 tablet_bootstrap.cc:492] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:16.848121 2282 raft_consensus.cc:348] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:16.848316 2282 raft_consensus.cc:374] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.848398 2282 raft_consensus.cc:729] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.848547 2282 consensus_queue.cc:260] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:16.848912 2288 tablet_bootstrap.cc:654] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.849361 2282 ts_tablet_manager.cc:1398] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:16.849575 2282 tablet_bootstrap.cc:492] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:16.850286 2288 tablet_bootstrap.cc:492] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:16.850382 2288 ts_tablet_manager.cc:1367] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:16.850461 2282 tablet_bootstrap.cc:654] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.850790 2288 raft_consensus.cc:348] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.850872 2288 raft_consensus.cc:374] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.850893 2288 raft_consensus.cc:729] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.851001 2288 consensus_queue.cc:260] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.851300 2288 ts_tablet_manager.cc:1398] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:16.851502 2288 tablet_bootstrap.cc:492] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:16.851567 2282 tablet_bootstrap.cc:492] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:16.851634 2282 ts_tablet_manager.cc:1367] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:16.852094 2282 raft_consensus.cc:348] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.852202 2282 raft_consensus.cc:374] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.852236 2282 raft_consensus.cc:729] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.852344 2282 consensus_queue.cc:260] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.852466 2288 tablet_bootstrap.cc:654] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.852980 2282 ts_tablet_manager.cc:1398] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:16.853188 2282 tablet_bootstrap.cc:492] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:16.854223 2282 tablet_bootstrap.cc:654] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:16.855520 2282 tablet_bootstrap.cc:492] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:16.855607 2282 ts_tablet_manager.cc:1367] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:16.855993 2288 tablet_bootstrap.cc:492] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:16.856104 2288 ts_tablet_manager.cc:1367] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:16.856101 2282 raft_consensus.cc:348] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.856235 2282 raft_consensus.cc:374] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.856287 2282 raft_consensus.cc:729] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.856458 2282 consensus_queue.cc:260] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.856534 2288 raft_consensus.cc:348] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.856663 2288 raft_consensus.cc:374] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:16.856711 2288 raft_consensus.cc:729] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:16.856859 2282 ts_tablet_manager.cc:1398] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:16.856885 2288 consensus_queue.cc:260] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.857187 2288 ts_tablet_manager.cc:1398] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:16.968887 2042 raft_consensus.cc:482] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:16.969040 2042 raft_consensus.cc:504] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.969465 2042 leader_election.cc:288] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:16.969909 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34e6f7677fbe40568b51a023be4c1ccd" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:16.969933 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34e6f7677fbe40568b51a023be4c1ccd" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:16.970055 408 raft_consensus.cc:2454] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:16.970101 672 raft_consensus.cc:2454] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:16.970387 738 leader_election.cc:302] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:16.970638 2042 raft_consensus.cc:2781] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:16.970690 2042 raft_consensus.cc:482] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:16.970726 2042 raft_consensus.cc:3037] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.971725 2042 raft_consensus.cc:504] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.972071 2042 leader_election.cc:288] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:16.972452 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34e6f7677fbe40568b51a023be4c1ccd" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:16.972608 408 raft_consensus.cc:3037] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.972584 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34e6f7677fbe40568b51a023be4c1ccd" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:16.972708 672 raft_consensus.cc:3037] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.973438 408 raft_consensus.cc:2454] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:16.973757 740 leader_election.cc:302] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:16.973801 672 raft_consensus.cc:2454] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:16.973910 2042 raft_consensus.cc:2781] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:16.974117 2042 raft_consensus.cc:686] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:16.974269 2042 consensus_queue.cc:237] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:16.976176 32725 catalog_manager.cc:5484] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:16.995764 2259 raft_consensus.cc:482] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:16.995944 2259 raft_consensus.cc:504] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:16.996402 2259 leader_election.cc:288] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:16.996793 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95cf29cac53140ac8ab67ad52b95a886" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:16.996836 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95cf29cac53140ac8ab67ad52b95a886" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:16.996951 408 raft_consensus.cc:2454] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:16.996937 542 raft_consensus.cc:2454] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:16.997291 608 leader_election.cc:302] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:16.997498 2259 raft_consensus.cc:2781] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:16.997557 2259 raft_consensus.cc:482] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:16.997606 2259 raft_consensus.cc:3037] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.998498 2259 raft_consensus.cc:504] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:16.998818 2259 leader_election.cc:288] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:16.999186 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95cf29cac53140ac8ab67ad52b95a886" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:16.999176 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95cf29cac53140ac8ab67ad52b95a886" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:16.999301 408 raft_consensus.cc:3037] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:16.999310 542 raft_consensus.cc:3037] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.000099 408 raft_consensus.cc:2454] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:17.000409 608 leader_election.cc:302] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:17.000443 542 raft_consensus.cc:2454] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:17.000571 2259 raft_consensus.cc:2781] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:17.000655 2259 raft_consensus.cc:686] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:17.000802 2259 consensus_queue.cc:237] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:17.003166 32725 catalog_manager.cc:5484] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:17.051643 2270 raft_consensus.cc:482] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:17.051781 2270 raft_consensus.cc:504] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:17.052242 2270 leader_election.cc:288] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:17.052614 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb75f88d4c7e42f787118144e9f6c430" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:17.052750 542 raft_consensus.cc:2454] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:17.052999 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb75f88d4c7e42f787118144e9f6c430" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:17.053128 804 raft_consensus.cc:2454] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:17.053135 341 leader_election.cc:302] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:17.053330 2270 raft_consensus.cc:2781] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:17.053398 2270 raft_consensus.cc:482] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:17.053444 2270 raft_consensus.cc:3037] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.054698 2270 raft_consensus.cc:504] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:17.055022 2270 leader_election.cc:288] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:17.055419 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb75f88d4c7e42f787118144e9f6c430" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:17.055492 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb75f88d4c7e42f787118144e9f6c430" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:17.055583 542 raft_consensus.cc:3037] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.055598 804 raft_consensus.cc:3037] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.056617 804 raft_consensus.cc:2454] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:17.056617 542 raft_consensus.cc:2454] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:17.056886 341 leader_election.cc:302] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:17.057194 2270 raft_consensus.cc:2781] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:17.057286 2270 raft_consensus.cc:686] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:17.057448 2270 consensus_queue.cc:237] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:17.059301 32725 catalog_manager.cc:5484] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } }
W20240502 14:00:17.187516 2298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:17.187904 2298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:17.191423 2298 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_12'
I20240502 14:00:17.277698 674 raft_consensus.cc:1261] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.278278 2042 consensus_queue.cc:1035] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.278383 804 raft_consensus.cc:1261] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.278873 541 raft_consensus.cc:1261] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.279240 408 raft_consensus.cc:1261] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.279531 2270 consensus_queue.cc:1035] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.282310 2294 consensus_queue.cc:1035] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.284238 409 raft_consensus.cc:1261] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.284602 2044 consensus_queue.cc:1035] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.284672 2141 consensus_queue.cc:1035] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.288152 541 raft_consensus.cc:1261] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.292527 2259 consensus_queue.cc:1035] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.322520 2254 mvcc.cc:205] Tried to move back new op lower bound from 7023240877175062528 to 7023240875927777280. Current Snapshot: MvccSnapshot[applied={T|T < 7023240877175062528}]
I20240502 14:00:17.328981 2261 mvcc.cc:205] Tried to move back new op lower bound from 7023240877163982848 to 7023240876268720128. Current Snapshot: MvccSnapshot[applied={T|T < 7023240877163982848}]
INSERT report
rows total: 2000
time total: 104.592 ms
time per row: 0.0522962 ms
I20240502 14:00:17.343546 32729 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:17.349324 521 tablet_service.cc:1467] Processing CreateTablet for tablet be795fbe277b4e9fac8561d6589b7c05 (DEFAULT_TABLE table=rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:17.349543 654 tablet_service.cc:1467] Processing CreateTablet for tablet be795fbe277b4e9fac8561d6589b7c05 (DEFAULT_TABLE table=rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:17.349666 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be795fbe277b4e9fac8561d6589b7c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:17.349834 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be795fbe277b4e9fac8561d6589b7c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:17.350323 386 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdc7b131596496aa6779885fe5d9433 (DEFAULT_TABLE table=rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:17.350585 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdc7b131596496aa6779885fe5d9433. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:17.350687 520 tablet_service.cc:1467] Processing CreateTablet for tablet 567df9d664b7423785175fa385f83e53 (DEFAULT_TABLE table=rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:17.350934 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567df9d664b7423785175fa385f83e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:17.352078 522 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdc7b131596496aa6779885fe5d9433 (DEFAULT_TABLE table=rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:17.352277 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdc7b131596496aa6779885fe5d9433. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:17.352320 784 tablet_service.cc:1467] Processing CreateTablet for tablet be795fbe277b4e9fac8561d6589b7c05 (DEFAULT_TABLE table=rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:17.352577 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be795fbe277b4e9fac8561d6589b7c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:17.352635 2282 tablet_bootstrap.cc:492] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:17.353449 2282 tablet_bootstrap.cc:654] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:17.354493 2282 tablet_bootstrap.cc:492] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:17.354578 2282 ts_tablet_manager.cc:1367] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:17.354620 653 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdc7b131596496aa6779885fe5d9433 (DEFAULT_TABLE table=rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:17.354874 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdc7b131596496aa6779885fe5d9433. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:17.355072 785 tablet_service.cc:1467] Processing CreateTablet for tablet 567df9d664b7423785175fa385f83e53 (DEFAULT_TABLE table=rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:17.355077 2282 raft_consensus.cc:348] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:17.355211 2282 raft_consensus.cc:374] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:17.355243 2282 raft_consensus.cc:729] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:17.355327 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567df9d664b7423785175fa385f83e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:17.355376 2282 consensus_queue.cc:260] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:17.355638 2282 ts_tablet_manager.cc:1398] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:17.355638 2347 tablet_bootstrap.cc:492] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:17.356103 654 tablet_service.cc:1467] Processing CreateTablet for tablet 567df9d664b7423785175fa385f83e53 (DEFAULT_TABLE table=rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:17.356333 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567df9d664b7423785175fa385f83e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:17.356719 2347 tablet_bootstrap.cc:654] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:17.357440 2288 tablet_bootstrap.cc:492] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:17.358117 2347 tablet_bootstrap.cc:492] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:17.358201 2347 ts_tablet_manager.cc:1367] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:17.358246 2288 tablet_bootstrap.cc:654] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:17.358847 2347 raft_consensus.cc:348] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.358944 2347 raft_consensus.cc:374] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:17.358976 2347 raft_consensus.cc:729] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:17.359108 2347 consensus_queue.cc:260] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.359210 2288 tablet_bootstrap.cc:492] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:17.359270 2288 ts_tablet_manager.cc:1367] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:17.359400 2347 ts_tablet_manager.cc:1398] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:17.359573 2347 tablet_bootstrap.cc:492] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:17.359680 2288 raft_consensus.cc:348] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.359792 2288 raft_consensus.cc:374] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:17.359859 2288 raft_consensus.cc:729] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:17.359989 2288 consensus_queue.cc:260] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.360272 2288 ts_tablet_manager.cc:1398] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:17.360447 2288 tablet_bootstrap.cc:492] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:17.360654 2347 tablet_bootstrap.cc:654] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:17.361804 2348 tablet_bootstrap.cc:492] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:17.361934 2288 tablet_bootstrap.cc:654] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:17.362762 2347 tablet_bootstrap.cc:492] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:17.362841 2347 ts_tablet_manager.cc:1367] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:17.363305 2347 raft_consensus.cc:348] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:17.363364 2288 tablet_bootstrap.cc:492] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:17.363404 2347 raft_consensus.cc:374] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:17.363453 2288 ts_tablet_manager.cc:1367] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:17.363453 2347 raft_consensus.cc:729] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:17.363605 2347 consensus_queue.cc:260] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:17.363902 2347 ts_tablet_manager.cc:1398] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:17.363943 2288 raft_consensus.cc:348] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.364048 2288 raft_consensus.cc:374] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:17.364084 2288 raft_consensus.cc:729] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:17.364094 2347 tablet_bootstrap.cc:492] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:17.364197 2288 consensus_queue.cc:260] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.364451 2288 ts_tablet_manager.cc:1398] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:17.365164 2347 tablet_bootstrap.cc:654] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:17.365268 2348 tablet_bootstrap.cc:654] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:17.366740 2348 tablet_bootstrap.cc:492] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:17.366847 2348 ts_tablet_manager.cc:1367] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:17.367230 2348 raft_consensus.cc:348] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.367357 2348 raft_consensus.cc:374] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:17.367393 2348 raft_consensus.cc:729] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:17.367542 2348 consensus_queue.cc:260] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.367921 2348 ts_tablet_manager.cc:1398] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:17.368079 2348 tablet_bootstrap.cc:492] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:17.369081 2347 tablet_bootstrap.cc:492] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:17.369096 2348 tablet_bootstrap.cc:654] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:17.369246 2347 ts_tablet_manager.cc:1367] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:17.369753 2347 raft_consensus.cc:348] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.370613 2347 raft_consensus.cc:374] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:17.370702 2347 raft_consensus.cc:729] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:17.370867 2347 consensus_queue.cc:260] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.371265 2347 ts_tablet_manager.cc:1398] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:17.371899 2348 tablet_bootstrap.cc:492] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:17.371994 2348 ts_tablet_manager.cc:1367] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:17.372442 2348 raft_consensus.cc:348] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:17.372581 2348 raft_consensus.cc:374] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:17.372630 2348 raft_consensus.cc:729] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:17.372750 2348 consensus_queue.cc:260] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:17.373055 2348 ts_tablet_manager.cc:1398] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:17.373323 2348 tablet_bootstrap.cc:492] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:17.374323 2348 tablet_bootstrap.cc:654] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:17.375459 2348 tablet_bootstrap.cc:492] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:17.375538 2348 ts_tablet_manager.cc:1367] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:17.375934 2348 raft_consensus.cc:348] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.376032 2348 raft_consensus.cc:374] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:17.376073 2348 raft_consensus.cc:729] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:17.376188 2348 consensus_queue.cc:260] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.376513 2348 ts_tablet_manager.cc:1398] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:17.474447 2259 raft_consensus.cc:482] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:17.474592 2259 raft_consensus.cc:504] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.475041 2259 leader_election.cc:288] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:17.475419 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be795fbe277b4e9fac8561d6589b7c05" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:17.475430 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be795fbe277b4e9fac8561d6589b7c05" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:17.475550 804 raft_consensus.cc:2454] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:17.475623 542 raft_consensus.cc:2454] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:17.475903 609 leader_election.cc:302] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:17.476121 2259 raft_consensus.cc:2781] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:17.476176 2259 raft_consensus.cc:482] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:17.476217 2259 raft_consensus.cc:3037] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.477274 2259 raft_consensus.cc:504] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.477631 2259 leader_election.cc:288] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:17.477917 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be795fbe277b4e9fac8561d6589b7c05" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:17.477897 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be795fbe277b4e9fac8561d6589b7c05" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:17.478024 542 raft_consensus.cc:3037] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.478024 804 raft_consensus.cc:3037] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.478996 542 raft_consensus.cc:2454] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:17.478994 804 raft_consensus.cc:2454] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:17.479281 609 leader_election.cc:302] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:17.479435 2259 raft_consensus.cc:2781] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:17.479550 2259 raft_consensus.cc:686] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:17.479691 2259 consensus_queue.cc:237] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.481640 32733 catalog_manager.cc:5484] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:17.524955 2270 raft_consensus.cc:482] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:17.525099 2270 raft_consensus.cc:504] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:17.525460 2270 leader_election.cc:288] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:17.525907 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdc7b131596496aa6779885fe5d9433" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:17.525976 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdc7b131596496aa6779885fe5d9433" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:17.526043 674 raft_consensus.cc:2454] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:17.526156 542 raft_consensus.cc:2454] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:17.526355 341 leader_election.cc:302] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:17.526531 2270 raft_consensus.cc:2781] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:17.526593 2270 raft_consensus.cc:482] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:17.526641 2270 raft_consensus.cc:3037] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.527469 2270 raft_consensus.cc:504] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:17.527772 2270 leader_election.cc:288] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:17.528154 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdc7b131596496aa6779885fe5d9433" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:17.528185 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdc7b131596496aa6779885fe5d9433" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:17.528255 674 raft_consensus.cc:3037] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.528303 542 raft_consensus.cc:3037] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.529072 674 raft_consensus.cc:2454] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:17.529088 542 raft_consensus.cc:2454] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:17.529364 341 leader_election.cc:302] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:17.529538 2270 raft_consensus.cc:2781] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:17.529620 2270 raft_consensus.cc:686] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:17.529968 2270 consensus_queue.cc:237] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:17.532032 32733 catalog_manager.cc:5484] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:17.674824 2259 raft_consensus.cc:482] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:17.674962 2259 raft_consensus.cc:504] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.675380 2259 leader_election.cc:288] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:17.675756 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567df9d664b7423785175fa385f83e53" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:17.675779 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567df9d664b7423785175fa385f83e53" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:17.675937 542 raft_consensus.cc:2454] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:17.675942 804 raft_consensus.cc:2454] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:17.676220 606 leader_election.cc:302] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:17.676390 2259 raft_consensus.cc:2781] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:17.676456 2259 raft_consensus.cc:482] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:17.676499 2259 raft_consensus.cc:3037] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.677707 2259 raft_consensus.cc:504] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.678076 2259 leader_election.cc:288] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:17.678419 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567df9d664b7423785175fa385f83e53" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:17.678534 542 raft_consensus.cc:3037] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.678488 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567df9d664b7423785175fa385f83e53" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:17.678637 804 raft_consensus.cc:3037] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:17.679302 542 raft_consensus.cc:2454] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:17.679502 804 raft_consensus.cc:2454] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:17.679605 606 leader_election.cc:302] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:17.679788 2259 raft_consensus.cc:2781] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:17.679901 2259 raft_consensus.cc:686] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:17.680083 2259 consensus_queue.cc:237] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:17.682118 32733 catalog_manager.cc:5484] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:17.847132 2362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:17.847462 2362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:17.851016 2362 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_13'
I20240502 14:00:17.946708 542 raft_consensus.cc:1261] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.946708 804 raft_consensus.cc:1261] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.947270 2259 consensus_queue.cc:1035] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.947794 2044 consensus_queue.cc:1035] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.947975 804 raft_consensus.cc:1261] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.948477 541 raft_consensus.cc:1261] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.951292 2044 consensus_queue.cc:1035] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.951856 2044 consensus_queue.cc:1035] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.963593 542 raft_consensus.cc:1261] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.963873 674 raft_consensus.cc:1261] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:17.964077 2270 consensus_queue.cc:1035] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.964286 2141 consensus_queue.cc:1035] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:17.987862 2394 mvcc.cc:205] Tried to move back new op lower bound from 7023240879969710080 to 7023240878203928576. Current Snapshot: MvccSnapshot[applied={T|T < 7023240879969710080}]
INSERT report
rows total: 2000
time total: 129.045 ms
time per row: 0.0645226 ms
I20240502 14:00:18.025137 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:18.031052 520 tablet_service.cc:1467] Processing CreateTablet for tablet 73e11a7fc3cb4b96924b6d705227f438 (DEFAULT_TABLE table=rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.031224 386 tablet_service.cc:1467] Processing CreateTablet for tablet 73e11a7fc3cb4b96924b6d705227f438 (DEFAULT_TABLE table=rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.031273 387 tablet_service.cc:1467] Processing CreateTablet for tablet a2dfd860f1164db198b890c2f5827051 (DEFAULT_TABLE table=rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.031407 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e11a7fc3cb4b96924b6d705227f438. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.031481 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e11a7fc3cb4b96924b6d705227f438. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.031642 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2dfd860f1164db198b890c2f5827051. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.031934 785 tablet_service.cc:1467] Processing CreateTablet for tablet 73e11a7fc3cb4b96924b6d705227f438 (DEFAULT_TABLE table=rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.032186 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e11a7fc3cb4b96924b6d705227f438. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.034178 386 tablet_service.cc:1467] Processing CreateTablet for tablet a286ed72ded04c2bb113066c7f574adf (DEFAULT_TABLE table=rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.034293 2424 tablet_bootstrap.cc:492] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:18.034418 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a286ed72ded04c2bb113066c7f574adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.034615 784 tablet_service.cc:1467] Processing CreateTablet for tablet a2dfd860f1164db198b890c2f5827051 (DEFAULT_TABLE table=rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.034865 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2dfd860f1164db198b890c2f5827051. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.035264 2424 tablet_bootstrap.cc:654] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.036000 2425 tablet_bootstrap.cc:492] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:18.036929 2424 tablet_bootstrap.cc:492] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:18.037037 2424 ts_tablet_manager.cc:1367] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:18.037549 2424 raft_consensus.cc:348] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.037655 2424 raft_consensus.cc:374] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.037685 2424 raft_consensus.cc:729] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.037708 654 tablet_service.cc:1467] Processing CreateTablet for tablet a2dfd860f1164db198b890c2f5827051 (DEFAULT_TABLE table=rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.037765 520 tablet_service.cc:1467] Processing CreateTablet for tablet a286ed72ded04c2bb113066c7f574adf (DEFAULT_TABLE table=rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.037807 2424 consensus_queue.cc:260] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.037972 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2dfd860f1164db198b890c2f5827051. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.038013 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a286ed72ded04c2bb113066c7f574adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.038105 2424 ts_tablet_manager.cc:1398] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:18.038718 2425 tablet_bootstrap.cc:654] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.039898 2424 tablet_bootstrap.cc:492] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:18.040846 2424 tablet_bootstrap.cc:654] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.041304 2425 tablet_bootstrap.cc:492] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:18.041436 2425 ts_tablet_manager.cc:1367] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:18.042074 2424 tablet_bootstrap.cc:492] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:18.042150 2424 ts_tablet_manager.cc:1367] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:18.042030 2425 raft_consensus.cc:348] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.042346 2425 raft_consensus.cc:374] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.042421 2425 raft_consensus.cc:729] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.042601 2425 consensus_queue.cc:260] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.042635 2424 raft_consensus.cc:348] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:18.042742 2424 raft_consensus.cc:374] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.042781 2424 raft_consensus.cc:729] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.042884 2424 consensus_queue.cc:260] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:18.042888 784 tablet_service.cc:1467] Processing CreateTablet for tablet a286ed72ded04c2bb113066c7f574adf (DEFAULT_TABLE table=rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.043157 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a286ed72ded04c2bb113066c7f574adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.043165 2425 ts_tablet_manager.cc:1398] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:18.043177 2424 ts_tablet_manager.cc:1398] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:18.043495 2426 tablet_bootstrap.cc:492] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:18.044003 2425 tablet_bootstrap.cc:492] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:18.044346 2426 tablet_bootstrap.cc:654] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.044883 2425 tablet_bootstrap.cc:654] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.045892 2426 tablet_bootstrap.cc:492] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:18.045995 2426 ts_tablet_manager.cc:1367] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:18.046789 2425 tablet_bootstrap.cc:492] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:18.046876 2425 ts_tablet_manager.cc:1367] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:18.047294 2425 raft_consensus.cc:348] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:18.047394 2425 raft_consensus.cc:374] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.047426 2425 raft_consensus.cc:729] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.047531 2425 consensus_queue.cc:260] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:18.047853 2425 ts_tablet_manager.cc:1398] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:18.048089 2425 tablet_bootstrap.cc:492] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:18.049021 2425 tablet_bootstrap.cc:654] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.049211 2429 tablet_bootstrap.cc:492] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:18.046422 2426 raft_consensus.cc:348] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.050031 2426 raft_consensus.cc:374] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.050077 2426 raft_consensus.cc:729] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.050199 2426 consensus_queue.cc:260] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.051256 2425 tablet_bootstrap.cc:492] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:18.051294 2429 tablet_bootstrap.cc:654] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.051347 2425 ts_tablet_manager.cc:1367] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:18.051798 2425 raft_consensus.cc:348] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:18.051923 2425 raft_consensus.cc:374] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.051950 2425 raft_consensus.cc:729] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.052084 2425 consensus_queue.cc:260] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:18.052400 2426 ts_tablet_manager.cc:1398] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:00:18.052601 2426 tablet_bootstrap.cc:492] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:18.052740 2425 ts_tablet_manager.cc:1398] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:18.054955 2429 tablet_bootstrap.cc:492] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:18.055065 2429 ts_tablet_manager.cc:1367] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:18.055467 2429 raft_consensus.cc:348] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:18.055565 2429 raft_consensus.cc:374] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.055593 2429 raft_consensus.cc:729] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.055716 2429 consensus_queue.cc:260] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:18.055873 2426 tablet_bootstrap.cc:654] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.056010 2429 ts_tablet_manager.cc:1398] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:18.056964 2426 tablet_bootstrap.cc:492] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:18.057046 2426 ts_tablet_manager.cc:1367] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:18.057488 2426 raft_consensus.cc:348] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:18.057621 2426 raft_consensus.cc:374] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.057662 2426 raft_consensus.cc:729] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.057798 2426 consensus_queue.cc:260] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:18.058732 2426 ts_tablet_manager.cc:1398] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:18.059046 2426 tablet_bootstrap.cc:492] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:18.059953 2426 tablet_bootstrap.cc:654] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.072427 2426 tablet_bootstrap.cc:492] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:18.072548 2426 ts_tablet_manager.cc:1367] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20240502 14:00:18.072962 2426 raft_consensus.cc:348] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:18.073067 2426 raft_consensus.cc:374] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.073100 2426 raft_consensus.cc:729] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.073236 2426 consensus_queue.cc:260] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:18.073513 2426 ts_tablet_manager.cc:1398] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:18.096699 2423 raft_consensus.cc:482] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:18.096827 2423 raft_consensus.cc:504] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:18.097278 2423 leader_election.cc:288] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:18.097625 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a286ed72ded04c2bb113066c7f574adf" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:18.097643 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a286ed72ded04c2bb113066c7f574adf" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:18.097757 806 raft_consensus.cc:2454] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:18.097786 409 raft_consensus.cc:2454] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:18.098071 477 leader_election.cc:302] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:18.098270 2423 raft_consensus.cc:2781] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:18.098332 2423 raft_consensus.cc:482] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:18.098374 2423 raft_consensus.cc:3037] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.099229 2423 raft_consensus.cc:504] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:18.099570 2423 leader_election.cc:288] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:18.099912 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a286ed72ded04c2bb113066c7f574adf" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:18.099913 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a286ed72ded04c2bb113066c7f574adf" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:18.100028 806 raft_consensus.cc:3037] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.100033 409 raft_consensus.cc:3037] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.100798 409 raft_consensus.cc:2454] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:18.100832 806 raft_consensus.cc:2454] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:18.101099 476 leader_election.cc:302] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:18.101256 2423 raft_consensus.cc:2781] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:18.101348 2423 raft_consensus.cc:686] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:18.101496 2423 consensus_queue.cc:237] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:18.103545 32730 catalog_manager.cc:5484] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:18.153791 2259 raft_consensus.cc:482] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:18.153934 2259 raft_consensus.cc:504] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:18.154408 2259 leader_election.cc:288] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:18.154769 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2dfd860f1164db198b890c2f5827051" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:18.154913 409 raft_consensus.cc:2454] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:18.155241 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2dfd860f1164db198b890c2f5827051" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:18.155315 608 leader_election.cc:302] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:18.155398 806 raft_consensus.cc:2454] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:18.155484 2259 raft_consensus.cc:2781] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:18.155542 2259 raft_consensus.cc:482] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:18.155599 2259 raft_consensus.cc:3037] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.156527 2259 raft_consensus.cc:504] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:18.156814 2259 leader_election.cc:288] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:18.157177 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2dfd860f1164db198b890c2f5827051" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:18.157191 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2dfd860f1164db198b890c2f5827051" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:18.157302 806 raft_consensus.cc:3037] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.157302 409 raft_consensus.cc:3037] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.158226 806 raft_consensus.cc:2454] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:18.158226 409 raft_consensus.cc:2454] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:18.158524 608 leader_election.cc:302] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:18.158679 2259 raft_consensus.cc:2781] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:18.158767 2259 raft_consensus.cc:686] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:18.158905 2259 consensus_queue.cc:237] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:18.160439 2423 raft_consensus.cc:482] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:18.160573 2423 raft_consensus.cc:504] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.160950 2423 leader_election.cc:288] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:18.160974 32730 catalog_manager.cc:5484] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:18.161252 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e11a7fc3cb4b96924b6d705227f438" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:18.161358 806 raft_consensus.cc:2454] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:18.161634 477 leader_election.cc:302] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:18.161702 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e11a7fc3cb4b96924b6d705227f438" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:18.161804 2423 raft_consensus.cc:2781] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:18.161816 409 raft_consensus.cc:2454] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:18.161855 2423 raft_consensus.cc:482] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:18.161881 2423 raft_consensus.cc:3037] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.162817 2423 raft_consensus.cc:504] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.163472 2423 leader_election.cc:288] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:18.163532 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e11a7fc3cb4b96924b6d705227f438" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:18.163535 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e11a7fc3cb4b96924b6d705227f438" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:18.163647 409 raft_consensus.cc:3037] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.163655 806 raft_consensus.cc:3037] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.164602 409 raft_consensus.cc:2454] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:18.164602 806 raft_consensus.cc:2454] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:18.164886 477 leader_election.cc:302] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:18.165043 2423 raft_consensus.cc:2781] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:18.165131 2423 raft_consensus.cc:686] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:18.165270 2423 consensus_queue.cc:237] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.167220 32730 catalog_manager.cc:5484] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:18.312695 2440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:18.312965 2440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:18.316129 2440 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_14'
I20240502 14:00:18.408542 806 raft_consensus.cc:1261] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:18.409711 409 raft_consensus.cc:1261] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:18.409806 408 raft_consensus.cc:1261] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:18.410243 2423 consensus_queue.cc:1035] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:18.410377 408 raft_consensus.cc:1261] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:18.410329 2187 consensus_queue.cc:1035] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:18.410552 2244 consensus_queue.cc:1035] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:18.410959 804 raft_consensus.cc:1261] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:18.410991 806 raft_consensus.cc:1261] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:18.411518 2259 consensus_queue.cc:1035] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:18.412550 2259 consensus_queue.cc:1035] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:18.414696 2244 consensus_queue.cc:1035] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:18.439066 2400 mvcc.cc:205] Tried to move back new op lower bound from 7023240881801789440 to 7023240880545259520. Current Snapshot: MvccSnapshot[applied={T|T < 7023240881801789440}]
INSERT report
rows total: 2000
time total: 120.99 ms
time per row: 0.0604952 ms
I20240502 14:00:18.484094 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:18.490640 654 tablet_service.cc:1467] Processing CreateTablet for tablet 02a36ffbdd0f4d68953aa441c04b42eb (DEFAULT_TABLE table=rebalance_test_table_15 [id=d4238514953942f2a3e7af65ce92b025]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.490640 520 tablet_service.cc:1467] Processing CreateTablet for tablet 02a36ffbdd0f4d68953aa441c04b42eb (DEFAULT_TABLE table=rebalance_test_table_15 [id=d4238514953942f2a3e7af65ce92b025]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.490965 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a36ffbdd0f4d68953aa441c04b42eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.490978 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a36ffbdd0f4d68953aa441c04b42eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.491438 386 tablet_service.cc:1467] Processing CreateTablet for tablet 02a36ffbdd0f4d68953aa441c04b42eb (DEFAULT_TABLE table=rebalance_test_table_15 [id=d4238514953942f2a3e7af65ce92b025]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.491703 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a36ffbdd0f4d68953aa441c04b42eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.493651 2429 tablet_bootstrap.cc:492] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:18.493876 387 tablet_service.cc:1467] Processing CreateTablet for tablet 9486274cd39b4de18c37ab9f71f261c2 (DEFAULT_TABLE table=rebalance_test_table_15 [id=d4238514953942f2a3e7af65ce92b025]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.494096 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9486274cd39b4de18c37ab9f71f261c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.494558 2429 tablet_bootstrap.cc:654] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.495479 785 tablet_service.cc:1467] Processing CreateTablet for tablet ce8c8f96e8944bba88ad9b818e27c916 (DEFAULT_TABLE table=rebalance_test_table_15 [id=d4238514953942f2a3e7af65ce92b025]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.495760 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce8c8f96e8944bba88ad9b818e27c916. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.495479 784 tablet_service.cc:1467] Processing CreateTablet for tablet 9486274cd39b4de18c37ab9f71f261c2 (DEFAULT_TABLE table=rebalance_test_table_15 [id=d4238514953942f2a3e7af65ce92b025]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.495981 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9486274cd39b4de18c37ab9f71f261c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.495978 2425 tablet_bootstrap.cc:492] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:18.496795 2425 tablet_bootstrap.cc:654] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.498102 2426 tablet_bootstrap.cc:492] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:18.498328 654 tablet_service.cc:1467] Processing CreateTablet for tablet 9486274cd39b4de18c37ab9f71f261c2 (DEFAULT_TABLE table=rebalance_test_table_15 [id=d4238514953942f2a3e7af65ce92b025]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.498587 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9486274cd39b4de18c37ab9f71f261c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.498670 2425 tablet_bootstrap.cc:492] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:18.498744 2425 ts_tablet_manager.cc:1367] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:18.499025 2426 tablet_bootstrap.cc:654] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.499152 2425 raft_consensus.cc:348] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.499250 2425 raft_consensus.cc:374] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.499280 2425 raft_consensus.cc:729] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.499395 2425 consensus_queue.cc:260] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.499653 2425 ts_tablet_manager.cc:1398] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:18.499753 653 tablet_service.cc:1467] Processing CreateTablet for tablet ce8c8f96e8944bba88ad9b818e27c916 (DEFAULT_TABLE table=rebalance_test_table_15 [id=d4238514953942f2a3e7af65ce92b025]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.499807 2425 tablet_bootstrap.cc:492] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:18.500041 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce8c8f96e8944bba88ad9b818e27c916. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.500679 2425 tablet_bootstrap.cc:654] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.501767 2425 tablet_bootstrap.cc:492] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:18.501840 2425 ts_tablet_manager.cc:1367] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:18.502295 2425 raft_consensus.cc:348] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.502743 2425 raft_consensus.cc:374] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.502789 2425 raft_consensus.cc:729] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.502919 2425 consensus_queue.cc:260] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.503199 2425 ts_tablet_manager.cc:1398] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:18.503163 522 tablet_service.cc:1467] Processing CreateTablet for tablet ce8c8f96e8944bba88ad9b818e27c916 (DEFAULT_TABLE table=rebalance_test_table_15 [id=d4238514953942f2a3e7af65ce92b025]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:18.503441 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce8c8f96e8944bba88ad9b818e27c916. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.505553 2424 tablet_bootstrap.cc:492] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:18.506213 2429 tablet_bootstrap.cc:492] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:18.506315 2429 ts_tablet_manager.cc:1367] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 14:00:18.506474 2424 tablet_bootstrap.cc:654] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.506690 2429 raft_consensus.cc:348] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.506786 2429 raft_consensus.cc:374] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.506808 2429 raft_consensus.cc:729] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.506925 2429 consensus_queue.cc:260] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.507191 2429 ts_tablet_manager.cc:1398] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:18.507361 2429 tablet_bootstrap.cc:492] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:18.507586 2424 tablet_bootstrap.cc:492] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:18.507666 2424 ts_tablet_manager.cc:1367] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:18.508153 2424 raft_consensus.cc:348] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.508224 2429 tablet_bootstrap.cc:654] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.508241 2424 raft_consensus.cc:374] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.508265 2424 raft_consensus.cc:729] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.508399 2424 consensus_queue.cc:260] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.508682 2424 ts_tablet_manager.cc:1398] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:18.509229 2429 tablet_bootstrap.cc:492] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:18.509572 2429 ts_tablet_manager.cc:1367] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:18.509637 2423 raft_consensus.cc:482] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:18.509729 2423 raft_consensus.cc:504] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.510084 2426 tablet_bootstrap.cc:492] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:18.510115 2423 leader_election.cc:288] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:18.510193 2426 ts_tablet_manager.cc:1367] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.001s
I20240502 14:00:18.510327 2429 raft_consensus.cc:348] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.510663 2426 raft_consensus.cc:348] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.510773 2426 raft_consensus.cc:374] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.510808 2426 raft_consensus.cc:729] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.510860 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8c8f96e8944bba88ad9b818e27c916" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:18.510689 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8c8f96e8944bba88ad9b818e27c916" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:18.510926 2426 consensus_queue.cc:260] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.511204 2426 ts_tablet_manager.cc:1398] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:18.510457 2429 raft_consensus.cc:374] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.511317 2429 raft_consensus.cc:729] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
W20240502 14:00:18.511425 474 leader_election.cc:341] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:00:18.511516 2426 tablet_bootstrap.cc:492] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:18.511493 2429 consensus_queue.cc:260] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.511807 2429 ts_tablet_manager.cc:1398] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:18.512040 2429 tablet_bootstrap.cc:492] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
W20240502 14:00:18.512365 477 leader_election.cc:341] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:00:18.512449 477 leader_election.cc:302] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751; no voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:00:18.512501 2426 tablet_bootstrap.cc:654] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.513008 2429 tablet_bootstrap.cc:654] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.513164 2423 raft_consensus.cc:2726] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 14:00:18.513741 2426 tablet_bootstrap.cc:492] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:18.513860 2426 ts_tablet_manager.cc:1367] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:18.514348 2426 raft_consensus.cc:348] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.514448 2426 raft_consensus.cc:374] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.514479 2426 raft_consensus.cc:729] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.514595 2426 consensus_queue.cc:260] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.514649 2429 tablet_bootstrap.cc:492] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:18.514715 2429 ts_tablet_manager.cc:1367] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:18.514915 2426 ts_tablet_manager.cc:1398] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:18.515163 2429 raft_consensus.cc:348] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.515276 2429 raft_consensus.cc:374] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.515285 2424 tablet_bootstrap.cc:492] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:18.515309 2429 raft_consensus.cc:729] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.515420 2429 consensus_queue.cc:260] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.515710 2429 ts_tablet_manager.cc:1398] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:18.516322 2424 tablet_bootstrap.cc:654] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:18.517437 2424 tablet_bootstrap.cc:492] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:18.517541 2424 ts_tablet_manager.cc:1367] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:18.517987 2424 raft_consensus.cc:348] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.518110 2424 raft_consensus.cc:374] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:18.518143 2424 raft_consensus.cc:729] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:18.518299 2424 consensus_queue.cc:260] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.518652 2424 ts_tablet_manager.cc:1398] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:18.556902 2141 raft_consensus.cc:482] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:18.557049 2141 raft_consensus.cc:504] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.557480 2141 leader_election.cc:288] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:18.557869 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a36ffbdd0f4d68953aa441c04b42eb" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:18.557922 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a36ffbdd0f4d68953aa441c04b42eb" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:18.558027 674 raft_consensus.cc:2454] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:18.558079 542 raft_consensus.cc:2454] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:18.558321 341 leader_election.cc:302] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:18.558508 2141 raft_consensus.cc:2781] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:18.558575 2141 raft_consensus.cc:482] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:18.558605 2141 raft_consensus.cc:3037] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.559638 2141 raft_consensus.cc:504] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.560040 2141 leader_election.cc:288] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:18.560355 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a36ffbdd0f4d68953aa441c04b42eb" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:18.560489 674 raft_consensus.cc:3037] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.561079 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a36ffbdd0f4d68953aa441c04b42eb" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:18.561185 541 raft_consensus.cc:3037] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.561508 674 raft_consensus.cc:2454] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:18.561759 341 leader_election.cc:302] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:18.562069 541 raft_consensus.cc:2454] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:18.562626 2420 raft_consensus.cc:2781] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:18.562810 2420 raft_consensus.cc:686] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:18.562968 2420 consensus_queue.cc:237] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:18.564793 32731 catalog_manager.cc:5484] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:18.623857 2294 raft_consensus.cc:482] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:18.623989 2294 raft_consensus.cc:504] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.624372 2294 leader_election.cc:288] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:18.624701 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9486274cd39b4de18c37ab9f71f261c2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:18.624747 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9486274cd39b4de18c37ab9f71f261c2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:18.624858 674 raft_consensus.cc:2454] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:18.624871 409 raft_consensus.cc:2454] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:18.625142 740 leader_election.cc:302] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:18.625314 2294 raft_consensus.cc:2781] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:18.625388 2294 raft_consensus.cc:482] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:18.625417 2294 raft_consensus.cc:3037] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.626309 2294 raft_consensus.cc:504] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.626627 2294 leader_election.cc:288] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:18.626938 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9486274cd39b4de18c37ab9f71f261c2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:18.626968 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9486274cd39b4de18c37ab9f71f261c2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:18.627068 409 raft_consensus.cc:3037] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.627082 674 raft_consensus.cc:3037] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.628072 409 raft_consensus.cc:2454] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:18.628072 674 raft_consensus.cc:2454] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:18.628377 740 leader_election.cc:302] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:18.628535 2294 raft_consensus.cc:2781] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:18.628621 2294 raft_consensus.cc:686] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:18.628767 2294 consensus_queue.cc:237] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.630923 32731 catalog_manager.cc:5484] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:18.750160 2259 raft_consensus.cc:482] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:18.750293 2259 raft_consensus.cc:504] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.750640 2259 leader_election.cc:288] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:18.751081 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8c8f96e8944bba88ad9b818e27c916" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:18.751092 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8c8f96e8944bba88ad9b818e27c916" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:18.751240 541 raft_consensus.cc:2454] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:18.751240 806 raft_consensus.cc:2454] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:18.751536 609 leader_election.cc:302] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:18.751710 2259 raft_consensus.cc:2781] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:18.751778 2259 raft_consensus.cc:482] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:18.751879 2259 raft_consensus.cc:3037] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.752768 2259 raft_consensus.cc:504] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.753111 2259 leader_election.cc:288] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:18.753449 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8c8f96e8944bba88ad9b818e27c916" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:18.753451 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce8c8f96e8944bba88ad9b818e27c916" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:18.753551 541 raft_consensus.cc:3037] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.753563 806 raft_consensus.cc:3037] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:18.754325 541 raft_consensus.cc:2454] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:18.754565 806 raft_consensus.cc:2454] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:18.754654 606 leader_election.cc:302] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:18.754844 2259 raft_consensus.cc:2781] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:18.754976 2259 raft_consensus.cc:686] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:18.755136 2259 consensus_queue.cc:237] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:18.757228 32731 catalog_manager.cc:5484] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:18.922114 2490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:18.922401 2490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:18.925776 2490 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_15'
I20240502 14:00:19.022831 806 raft_consensus.cc:1261] T ce8c8f96e8944bba88ad9b818e27c916 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.023583 2259 consensus_queue.cc:1035] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:19.023926 541 raft_consensus.cc:1261] T ce8c8f96e8944bba88ad9b818e27c916 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.024339 2259 consensus_queue.cc:1035] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:19.041898 409 raft_consensus.cc:1261] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.042310 2294 consensus_queue.cc:1035] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:19.046479 674 raft_consensus.cc:1261] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.046924 2294 consensus_queue.cc:1035] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:19.056677 674 raft_consensus.cc:1261] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.058472 541 raft_consensus.cc:1261] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.058553 2486 consensus_queue.cc:1035] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:19.058960 2486 consensus_queue.cc:1035] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 139.966 ms
time per row: 0.0699832 ms
I20240502 14:00:19.112915 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:19.118712 387 tablet_service.cc:1467] Processing CreateTablet for tablet 68992bd623d848f390b36e093890fa70 (DEFAULT_TABLE table=rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.118711 520 tablet_service.cc:1467] Processing CreateTablet for tablet 68992bd623d848f390b36e093890fa70 (DEFAULT_TABLE table=rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.119048 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68992bd623d848f390b36e093890fa70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.119048 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68992bd623d848f390b36e093890fa70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.119576 653 tablet_service.cc:1467] Processing CreateTablet for tablet 68992bd623d848f390b36e093890fa70 (DEFAULT_TABLE table=rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.119576 654 tablet_service.cc:1467] Processing CreateTablet for tablet 12dc7df342664dbb9a50921736baf935 (DEFAULT_TABLE table=rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.119926 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68992bd623d848f390b36e093890fa70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.120254 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12dc7df342664dbb9a50921736baf935. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.121822 2553 tablet_bootstrap.cc:492] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:19.121776 522 tablet_service.cc:1467] Processing CreateTablet for tablet edad7a65b4114f348c4b5a5754213120 (DEFAULT_TABLE table=rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.122012 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edad7a65b4114f348c4b5a5754213120. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.122687 2554 tablet_bootstrap.cc:492] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:19.123344 2553 tablet_bootstrap.cc:654] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.123646 2554 tablet_bootstrap.cc:654] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.124374 785 tablet_service.cc:1467] Processing CreateTablet for tablet edad7a65b4114f348c4b5a5754213120 (DEFAULT_TABLE table=rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.124374 784 tablet_service.cc:1467] Processing CreateTablet for tablet 12dc7df342664dbb9a50921736baf935 (DEFAULT_TABLE table=rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.124673 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edad7a65b4114f348c4b5a5754213120. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.125102 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12dc7df342664dbb9a50921736baf935. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.126731 2557 tablet_bootstrap.cc:492] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:19.127569 387 tablet_service.cc:1467] Processing CreateTablet for tablet edad7a65b4114f348c4b5a5754213120 (DEFAULT_TABLE table=rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.127871 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edad7a65b4114f348c4b5a5754213120. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.128517 2552 tablet_bootstrap.cc:492] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:19.129318 2554 tablet_bootstrap.cc:492] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:19.129401 2554 ts_tablet_manager.cc:1367] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:19.129418 2552 tablet_bootstrap.cc:654] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.130110 2554 raft_consensus.cc:348] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:19.130187 386 tablet_service.cc:1467] Processing CreateTablet for tablet 12dc7df342664dbb9a50921736baf935 (DEFAULT_TABLE table=rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.130254 2554 raft_consensus.cc:374] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.130288 2554 raft_consensus.cc:729] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.130422 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12dc7df342664dbb9a50921736baf935. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.130416 2554 consensus_queue.cc:260] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:19.130525 2553 tablet_bootstrap.cc:492] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:19.130610 2553 ts_tablet_manager.cc:1367] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 14:00:19.130803 2554 ts_tablet_manager.cc:1398] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.130882 2557 tablet_bootstrap.cc:654] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.131052 2553 raft_consensus.cc:348] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:19.131148 2553 raft_consensus.cc:374] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.131418 2554 tablet_bootstrap.cc:492] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:19.132369 2554 tablet_bootstrap.cc:654] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.131178 2553 raft_consensus.cc:729] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.132902 2553 consensus_queue.cc:260] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:19.133306 2553 ts_tablet_manager.cc:1398] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:00:19.133586 2553 tablet_bootstrap.cc:492] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:19.134155 2552 tablet_bootstrap.cc:492] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:19.134256 2552 ts_tablet_manager.cc:1367] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:19.134518 2553 tablet_bootstrap.cc:654] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.134778 2552 raft_consensus.cc:348] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:19.135675 2553 tablet_bootstrap.cc:492] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:19.135717 2552 raft_consensus.cc:374] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.135761 2552 raft_consensus.cc:729] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.135757 2553 ts_tablet_manager.cc:1367] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:19.135931 2552 consensus_queue.cc:260] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:19.136229 2553 raft_consensus.cc:348] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.136312 2552 ts_tablet_manager.cc:1398] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:19.136323 2553 raft_consensus.cc:374] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.136356 2553 raft_consensus.cc:729] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.136523 2552 tablet_bootstrap.cc:492] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:19.136469 2553 consensus_queue.cc:260] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.136725 2553 ts_tablet_manager.cc:1398] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.136869 2553 tablet_bootstrap.cc:492] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:19.137310 2552 tablet_bootstrap.cc:654] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.137712 2553 tablet_bootstrap.cc:654] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.138468 2552 tablet_bootstrap.cc:492] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:19.138557 2552 ts_tablet_manager.cc:1367] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:19.138903 2553 tablet_bootstrap.cc:492] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:19.138988 2553 ts_tablet_manager.cc:1367] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:19.139005 2552 raft_consensus.cc:348] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.139104 2552 raft_consensus.cc:374] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.139137 2552 raft_consensus.cc:729] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.139266 2552 consensus_queue.cc:260] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.139444 2553 raft_consensus.cc:348] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.139544 2553 raft_consensus.cc:374] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.139577 2553 raft_consensus.cc:729] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.139595 2552 ts_tablet_manager.cc:1398] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.139681 2553 consensus_queue.cc:260] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.139976 2553 ts_tablet_manager.cc:1398] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.141901 2557 tablet_bootstrap.cc:492] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:19.142012 2557 ts_tablet_manager.cc:1367] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:00:19.142503 2557 raft_consensus.cc:348] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.142632 2557 raft_consensus.cc:374] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.142683 2557 raft_consensus.cc:729] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.142809 2557 consensus_queue.cc:260] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.143218 2557 ts_tablet_manager.cc:1398] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.143507 2557 tablet_bootstrap.cc:492] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:19.143692 2554 tablet_bootstrap.cc:492] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:19.144047 2554 ts_tablet_manager.cc:1367] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20240502 14:00:19.144521 2557 tablet_bootstrap.cc:654] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.144688 2554 raft_consensus.cc:348] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.144829 2554 raft_consensus.cc:374] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.144893 2554 raft_consensus.cc:729] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.145062 2554 consensus_queue.cc:260] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.145637 2557 tablet_bootstrap.cc:492] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:19.145730 2557 ts_tablet_manager.cc:1367] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:19.146124 2554 ts_tablet_manager.cc:1398] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:19.146186 2557 raft_consensus.cc:348] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.146334 2557 raft_consensus.cc:374] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.146399 2557 raft_consensus.cc:729] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.146569 2557 consensus_queue.cc:260] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.146961 2557 ts_tablet_manager.cc:1398] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.257786 2544 raft_consensus.cc:482] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:19.257907 2544 raft_consensus.cc:504] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.258280 2544 leader_election.cc:288] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:19.258625 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edad7a65b4114f348c4b5a5754213120" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:19.258694 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edad7a65b4114f348c4b5a5754213120" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:19.258772 409 raft_consensus.cc:2454] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:19.258836 541 raft_consensus.cc:2454] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:19.259126 738 leader_election.cc:302] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:19.259295 2544 raft_consensus.cc:2781] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:19.259356 2544 raft_consensus.cc:482] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:19.259423 2544 raft_consensus.cc:3037] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.260375 2544 raft_consensus.cc:504] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.260697 2544 leader_election.cc:288] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:19.261023 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edad7a65b4114f348c4b5a5754213120" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:19.261020 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edad7a65b4114f348c4b5a5754213120" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:19.261127 541 raft_consensus.cc:3037] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.261137 409 raft_consensus.cc:3037] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.261874 541 raft_consensus.cc:2454] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:19.261919 409 raft_consensus.cc:2454] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:19.262147 738 leader_election.cc:302] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:19.262343 2544 raft_consensus.cc:2781] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:19.262430 2544 raft_consensus.cc:686] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:19.262573 2544 consensus_queue.cc:237] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.264644 32731 catalog_manager.cc:5484] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:19.316375 2044 raft_consensus.cc:482] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:19.316494 2044 raft_consensus.cc:504] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:19.316823 2044 leader_election.cc:288] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:19.317183 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68992bd623d848f390b36e093890fa70" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:19.317198 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68992bd623d848f390b36e093890fa70" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:19.317307 541 raft_consensus.cc:2454] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:19.317363 409 raft_consensus.cc:2454] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:19.317584 606 leader_election.cc:302] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:19.317752 2044 raft_consensus.cc:2781] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:19.317799 2044 raft_consensus.cc:482] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:19.317845 2044 raft_consensus.cc:3037] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.318715 2044 raft_consensus.cc:504] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:19.319032 2044 leader_election.cc:288] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:19.319345 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68992bd623d848f390b36e093890fa70" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:19.319348 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68992bd623d848f390b36e093890fa70" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:19.319444 409 raft_consensus.cc:3037] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.319459 541 raft_consensus.cc:3037] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.320258 409 raft_consensus.cc:2454] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:19.320266 541 raft_consensus.cc:2454] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:19.320539 608 leader_election.cc:302] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:19.320716 2044 raft_consensus.cc:2781] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:19.320803 2044 raft_consensus.cc:686] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:19.320947 2044 consensus_queue.cc:237] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:19.322854 32731 catalog_manager.cc:5484] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:19.368443 2044 raft_consensus.cc:482] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:19.368562 2044 raft_consensus.cc:504] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.368965 2044 leader_election.cc:288] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:19.369319 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12dc7df342664dbb9a50921736baf935" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:19.369451 804 raft_consensus.cc:2454] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:19.369439 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12dc7df342664dbb9a50921736baf935" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:19.369601 409 raft_consensus.cc:2454] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:19.369798 609 leader_election.cc:302] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:19.369951 2044 raft_consensus.cc:2781] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:19.370020 2044 raft_consensus.cc:482] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:19.370060 2044 raft_consensus.cc:3037] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.370867 2044 raft_consensus.cc:504] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.371212 2044 leader_election.cc:288] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:19.371522 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12dc7df342664dbb9a50921736baf935" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:19.371551 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12dc7df342664dbb9a50921736baf935" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:19.371625 804 raft_consensus.cc:3037] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.371655 409 raft_consensus.cc:3037] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.372400 409 raft_consensus.cc:2454] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:19.372407 804 raft_consensus.cc:2454] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:19.372699 609 leader_election.cc:302] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:19.372844 2044 raft_consensus.cc:2781] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:19.372934 2044 raft_consensus.cc:686] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:19.373078 2044 consensus_queue.cc:237] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.375066 32731 catalog_manager.cc:5484] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:19.538414 2568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:19.538671 2568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:19.541790 2568 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_16'
I20240502 14:00:19.624599 806 raft_consensus.cc:1261] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.624603 409 raft_consensus.cc:1261] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.625207 2259 consensus_queue.cc:1035] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:19.625644 541 raft_consensus.cc:1261] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.626065 406 raft_consensus.cc:1261] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.626298 2472 consensus_queue.cc:1035] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:19.626571 2472 consensus_queue.cc:1035] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:19.628266 2606 consensus_queue.cc:1035] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:19.629055 409 raft_consensus.cc:1261] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.631186 2259 consensus_queue.cc:1035] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:19.635421 541 raft_consensus.cc:1261] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:19.636071 2044 consensus_queue.cc:1035] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 104.387 ms
time per row: 0.0521937 ms
I20240502 14:00:19.698789 32734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:19.705323 522 tablet_service.cc:1467] Processing CreateTablet for tablet e5bed83e31be4129a19eb8c347514b80 (DEFAULT_TABLE table=rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.705677 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5bed83e31be4129a19eb8c347514b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.706068 784 tablet_service.cc:1467] Processing CreateTablet for tablet e5bed83e31be4129a19eb8c347514b80 (DEFAULT_TABLE table=rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.706173 654 tablet_service.cc:1467] Processing CreateTablet for tablet e5bed83e31be4129a19eb8c347514b80 (DEFAULT_TABLE table=rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.706333 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5bed83e31be4129a19eb8c347514b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.706372 653 tablet_service.cc:1467] Processing CreateTablet for tablet 5f321621da5d481b8d1d2b25ddf4b395 (DEFAULT_TABLE table=rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.706455 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5bed83e31be4129a19eb8c347514b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.706575 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f321621da5d481b8d1d2b25ddf4b395. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.708127 386 tablet_service.cc:1467] Processing CreateTablet for tablet 987eabba58cd472696e64f30db468c71 (DEFAULT_TABLE table=rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.708422 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987eabba58cd472696e64f30db468c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.708712 2634 tablet_bootstrap.cc:492] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:19.709129 2635 tablet_bootstrap.cc:492] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:19.709327 522 tablet_service.cc:1467] Processing CreateTablet for tablet 5f321621da5d481b8d1d2b25ddf4b395 (DEFAULT_TABLE table=rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.709589 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f321621da5d481b8d1d2b25ddf4b395. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.709625 784 tablet_service.cc:1467] Processing CreateTablet for tablet 5f321621da5d481b8d1d2b25ddf4b395 (DEFAULT_TABLE table=rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.709841 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f321621da5d481b8d1d2b25ddf4b395. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.710763 2636 tablet_bootstrap.cc:492] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:19.711076 652 tablet_service.cc:1467] Processing CreateTablet for tablet 987eabba58cd472696e64f30db468c71 (DEFAULT_TABLE table=rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.711719 520 tablet_service.cc:1467] Processing CreateTablet for tablet 987eabba58cd472696e64f30db468c71 (DEFAULT_TABLE table=rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:19.711989 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987eabba58cd472696e64f30db468c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.712134 652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987eabba58cd472696e64f30db468c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:19.712855 2634 tablet_bootstrap.cc:654] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.713479 2635 tablet_bootstrap.cc:654] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.713796 2636 tablet_bootstrap.cc:654] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.714740 2635 tablet_bootstrap.cc:492] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:19.714869 2635 ts_tablet_manager.cc:1367] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:19.714913 2633 tablet_bootstrap.cc:492] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:19.715317 2635 raft_consensus.cc:348] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:19.715442 2635 raft_consensus.cc:374] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.715490 2635 raft_consensus.cc:729] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.715605 2635 consensus_queue.cc:260] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:19.715917 2635 ts_tablet_manager.cc:1398] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.716369 2633 tablet_bootstrap.cc:654] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.716857 2636 tablet_bootstrap.cc:492] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:19.716857 2634 tablet_bootstrap.cc:492] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:19.716946 2634 ts_tablet_manager.cc:1367] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:19.716946 2636 ts_tablet_manager.cc:1367] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:19.717342 2635 tablet_bootstrap.cc:492] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:19.717397 2636 raft_consensus.cc:348] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.717397 2634 raft_consensus.cc:348] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:19.717501 2636 raft_consensus.cc:374] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.717509 2634 raft_consensus.cc:374] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.717540 2636 raft_consensus.cc:729] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.717543 2634 raft_consensus.cc:729] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.717656 2636 consensus_queue.cc:260] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.717656 2634 consensus_queue.cc:260] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:19.717922 2636 ts_tablet_manager.cc:1398] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:19.717921 2634 ts_tablet_manager.cc:1398] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.718098 2634 tablet_bootstrap.cc:492] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:19.718251 2635 tablet_bootstrap.cc:654] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.718978 2634 tablet_bootstrap.cc:654] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.719717 2635 tablet_bootstrap.cc:492] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:19.720467 2635 ts_tablet_manager.cc:1367] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:19.720506 2633 tablet_bootstrap.cc:492] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:19.720604 2633 ts_tablet_manager.cc:1367] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:19.721082 2633 raft_consensus.cc:348] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:19.721213 2633 raft_consensus.cc:374] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.721254 2633 raft_consensus.cc:729] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.721376 2633 consensus_queue.cc:260] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:19.721488 2634 tablet_bootstrap.cc:492] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:19.721573 2634 ts_tablet_manager.cc:1367] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:19.721134 2635 raft_consensus.cc:348] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.721675 2633 ts_tablet_manager.cc:1398] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.721709 2635 raft_consensus.cc:374] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.721843 2633 tablet_bootstrap.cc:492] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:19.722039 2634 raft_consensus.cc:348] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.722143 2634 raft_consensus.cc:374] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.721751 2635 raft_consensus.cc:729] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.722180 2634 raft_consensus.cc:729] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.722292 2634 consensus_queue.cc:260] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.722330 2635 consensus_queue.cc:260] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.722582 2634 ts_tablet_manager.cc:1398] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.722610 2635 ts_tablet_manager.cc:1398] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:19.722754 2635 tablet_bootstrap.cc:492] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:19.722803 2633 tablet_bootstrap.cc:654] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.723624 2635 tablet_bootstrap.cc:654] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.726300 2633 tablet_bootstrap.cc:492] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:19.726393 2633 ts_tablet_manager.cc:1367] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:19.726847 2633 raft_consensus.cc:348] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.726977 2633 raft_consensus.cc:374] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.727011 2633 raft_consensus.cc:729] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.727142 2633 consensus_queue.cc:260] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.727463 2633 ts_tablet_manager.cc:1398] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.727639 2633 tablet_bootstrap.cc:492] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:19.728263 2635 tablet_bootstrap.cc:492] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:19.728415 2635 ts_tablet_manager.cc:1367] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:19.728941 2633 tablet_bootstrap.cc:654] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:19.728958 2635 raft_consensus.cc:348] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.729079 2635 raft_consensus.cc:374] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.729110 2635 raft_consensus.cc:729] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.729288 2635 consensus_queue.cc:260] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.729950 2635 ts_tablet_manager.cc:1398] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:19.730460 2633 tablet_bootstrap.cc:492] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:19.730551 2633 ts_tablet_manager.cc:1367] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:19.730909 2633 raft_consensus.cc:348] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.731010 2633 raft_consensus.cc:374] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:19.731032 2633 raft_consensus.cc:729] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:19.731133 2633 consensus_queue.cc:260] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.731427 2633 ts_tablet_manager.cc:1398] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:19.733341 2259 raft_consensus.cc:482] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:19.733433 2259 raft_consensus.cc:504] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:19.733843 2259 leader_election.cc:288] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:19.734148 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:19.734256 542 raft_consensus.cc:2454] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:19.734277 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:19.734418 806 raft_consensus.cc:2454] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:19.734705 609 leader_election.cc:302] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:19.734894 2259 raft_consensus.cc:2781] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:19.734952 2259 raft_consensus.cc:482] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:19.734980 2259 raft_consensus.cc:3037] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.735713 2259 raft_consensus.cc:504] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:19.736042 2259 leader_election.cc:288] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:19.736341 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:19.736383 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:19.736464 542 raft_consensus.cc:3037] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.736502 806 raft_consensus.cc:3037] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.737246 542 raft_consensus.cc:2454] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:19.737257 806 raft_consensus.cc:2454] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:19.737551 606 leader_election.cc:302] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:19.737702 2259 raft_consensus.cc:2781] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:19.737787 2259 raft_consensus.cc:686] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:19.737936 2259 consensus_queue.cc:237] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:19.739943 32734 catalog_manager.cc:5484] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:19.749889 2259 raft_consensus.cc:482] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:19.750005 2259 raft_consensus.cc:504] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.750370 2259 leader_election.cc:288] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:19.750664 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f321621da5d481b8d1d2b25ddf4b395" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:19.750722 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f321621da5d481b8d1d2b25ddf4b395" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:19.750806 806 raft_consensus.cc:2454] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:19.750845 542 raft_consensus.cc:2454] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:19.751065 609 leader_election.cc:302] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:19.751283 2259 raft_consensus.cc:2781] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:19.751330 2259 raft_consensus.cc:482] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:19.751363 2259 raft_consensus.cc:3037] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.752213 2259 raft_consensus.cc:504] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.752517 2259 leader_election.cc:288] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:19.752861 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f321621da5d481b8d1d2b25ddf4b395" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:19.752908 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f321621da5d481b8d1d2b25ddf4b395" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:19.753002 806 raft_consensus.cc:3037] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.753046 542 raft_consensus.cc:3037] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.753809 806 raft_consensus.cc:2454] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:19.753904 542 raft_consensus.cc:2454] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:19.754209 606 leader_election.cc:302] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:19.754443 2259 raft_consensus.cc:2781] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:19.754842 2259 raft_consensus.cc:686] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:19.754994 2259 consensus_queue.cc:237] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:19.757747 32734 catalog_manager.cc:5484] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:19.847569 2187 raft_consensus.cc:482] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:19.847728 2187 raft_consensus.cc:504] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.847980 2259 raft_consensus.cc:482] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:19.848088 2259 raft_consensus.cc:504] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.848484 2259 leader_election.cc:288] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:19.848773 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987eabba58cd472696e64f30db468c71" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:19.848891 674 raft_consensus.cc:2454] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:19.848901 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987eabba58cd472696e64f30db468c71" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:19.849009 542 raft_consensus.cc:2454] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:19.849370 606 leader_election.cc:302] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:19.849416 474 leader_election.cc:302] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:19.849574 2615 raft_consensus.cc:2781] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:19.849623 2615 raft_consensus.cc:482] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:19.849684 2615 raft_consensus.cc:3037] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.849767 2605 raft_consensus.cc:2781] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:19.849830 2605 raft_consensus.cc:482] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:19.849892 2605 raft_consensus.cc:3037] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.850713 2605 raft_consensus.cc:504] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.850715 2615 raft_consensus.cc:504] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.851030 2605 leader_election.cc:288] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:19.851045 2615 leader_election.cc:288] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:19.851395 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987eabba58cd472696e64f30db468c71" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:19.851533 674 raft_consensus.cc:2379] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c351a89a5fe34e788b9d8064fbffa751 in current term 1: Already voted for candidate 58107b611f8f475a910a6fd54dfd1fb9 in this term.
I20240502 14:00:19.852063 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987eabba58cd472696e64f30db468c71" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:19.852209 541 raft_consensus.cc:2379] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 58107b611f8f475a910a6fd54dfd1fb9 in current term 1: Already voted for candidate c351a89a5fe34e788b9d8064fbffa751 in this term.
I20240502 14:00:19.852456 2187 leader_election.cc:288] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:19.853152 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987eabba58cd472696e64f30db468c71" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:19.853247 408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987eabba58cd472696e64f30db468c71" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:19.853281 405 raft_consensus.cc:3037] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:19.853147 406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987eabba58cd472696e64f30db468c71" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:19.853129 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987eabba58cd472696e64f30db468c71" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:19.853811 476 leader_election.cc:302] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751; no voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e
I20240502 14:00:19.853984 2187 raft_consensus.cc:2726] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240502 14:00:19.854312 405 raft_consensus.cc:2454] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:19.854579 608 leader_election.cc:302] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters: c351a89a5fe34e788b9d8064fbffa751
I20240502 14:00:19.854758 2259 raft_consensus.cc:2781] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:19.854868 2259 raft_consensus.cc:686] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:19.855010 2259 consensus_queue.cc:237] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:19.857362 32734 catalog_manager.cc:5484] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:19.983712 2649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:19.984079 2649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:19.987463 2649 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_17'
I20240502 14:00:20.071363 806 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.071420 541 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.071880 2606 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:20.072160 405 raft_consensus.cc:1261] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.072743 2606 consensus_queue.cc:1035] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:20.073000 541 raft_consensus.cc:1261] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.073025 2606 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:20.073590 540 raft_consensus.cc:1261] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.076977 2259 consensus_queue.cc:1035] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:20.077286 2259 consensus_queue.cc:1035] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:20.090600 2612 mvcc.cc:205] Tried to move back new op lower bound from 7023240888606617600 to 7023240887318372352. Current Snapshot: MvccSnapshot[applied={T|T < 7023240888606617600}]
I20240502 14:00:20.095266 806 raft_consensus.cc:1261] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.099069 2605 consensus_queue.cc:1035] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:20.099886 2608 mvcc.cc:205] Tried to move back new op lower bound from 7023240888604049408 to 7023240887727489024. Current Snapshot: MvccSnapshot[applied={T|T < 7023240888604049408}]
INSERT report
rows total: 2000
time total: 111.568 ms
time per row: 0.0557842 ms
I20240502 14:00:20.144217 32727 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:20.149976 520 tablet_service.cc:1467] Processing CreateTablet for tablet 8e38b601db0c4a51aa08f1f94107331e (DEFAULT_TABLE table=rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.150100 784 tablet_service.cc:1467] Processing CreateTablet for tablet 8e38b601db0c4a51aa08f1f94107331e (DEFAULT_TABLE table=rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.150310 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e38b601db0c4a51aa08f1f94107331e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.150378 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e38b601db0c4a51aa08f1f94107331e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.150365 386 tablet_service.cc:1467] Processing CreateTablet for tablet 8e38b601db0c4a51aa08f1f94107331e (DEFAULT_TABLE table=rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.150612 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e38b601db0c4a51aa08f1f94107331e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.152047 2633 tablet_bootstrap.cc:492] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:20.152638 2634 tablet_bootstrap.cc:492] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:20.152834 2633 tablet_bootstrap.cc:654] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.153007 387 tablet_service.cc:1467] Processing CreateTablet for tablet 1db6563013284583925202aeedc631ed (DEFAULT_TABLE table=rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.153198 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db6563013284583925202aeedc631ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.153594 2634 tablet_bootstrap.cc:654] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.154067 2633 tablet_bootstrap.cc:492] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:20.154148 2633 ts_tablet_manager.cc:1367] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:20.154721 2634 tablet_bootstrap.cc:492] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:20.154628 2633 raft_consensus.cc:348] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.154770 2633 raft_consensus.cc:374] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.154819 2634 ts_tablet_manager.cc:1367] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:20.154824 2633 raft_consensus.cc:729] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.154974 2636 tablet_bootstrap.cc:492] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:20.154979 2633 consensus_queue.cc:260] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.155268 2633 ts_tablet_manager.cc:1398] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:20.155241 2634 raft_consensus.cc:348] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.155337 2634 raft_consensus.cc:374] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.155373 2634 raft_consensus.cc:729] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.155519 2634 consensus_queue.cc:260] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.155767 2636 tablet_bootstrap.cc:654] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.155777 2634 ts_tablet_manager.cc:1398] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:20.157356 652 tablet_service.cc:1467] Processing CreateTablet for tablet 1db6563013284583925202aeedc631ed (DEFAULT_TABLE table=rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.157513 520 tablet_service.cc:1467] Processing CreateTablet for tablet d10d62c409d542ed8f80b952ccc48826 (DEFAULT_TABLE table=rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.157637 652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db6563013284583925202aeedc631ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.157723 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d10d62c409d542ed8f80b952ccc48826. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.158458 784 tablet_service.cc:1467] Processing CreateTablet for tablet 1db6563013284583925202aeedc631ed (DEFAULT_TABLE table=rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.158684 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db6563013284583925202aeedc631ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.158746 785 tablet_service.cc:1467] Processing CreateTablet for tablet d10d62c409d542ed8f80b952ccc48826 (DEFAULT_TABLE table=rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.158952 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d10d62c409d542ed8f80b952ccc48826. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.159664 2633 tablet_bootstrap.cc:492] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:20.160650 2633 tablet_bootstrap.cc:654] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.160765 2635 tablet_bootstrap.cc:492] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:20.161002 2634 tablet_bootstrap.cc:492] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:20.161512 2635 tablet_bootstrap.cc:654] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.162030 2634 tablet_bootstrap.cc:654] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.162477 2633 tablet_bootstrap.cc:492] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:20.162557 2633 ts_tablet_manager.cc:1367] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:20.163239 2635 tablet_bootstrap.cc:492] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:20.163328 2635 ts_tablet_manager.cc:1367] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:20.163239 2633 raft_consensus.cc:348] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.163525 2633 raft_consensus.cc:374] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.163604 2633 raft_consensus.cc:729] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.163779 2633 consensus_queue.cc:260] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.163812 2635 raft_consensus.cc:348] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:20.163949 2635 raft_consensus.cc:374] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.163986 2635 raft_consensus.cc:729] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.164005 2634 tablet_bootstrap.cc:492] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:20.164073 2634 ts_tablet_manager.cc:1367] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:20.164112 2635 consensus_queue.cc:260] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:20.164382 2635 ts_tablet_manager.cc:1398] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:20.164521 2634 raft_consensus.cc:348] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.164621 2634 raft_consensus.cc:374] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.164649 2634 raft_consensus.cc:729] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.164755 2634 consensus_queue.cc:260] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.164865 2633 ts_tablet_manager.cc:1398] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:20.165105 2634 ts_tablet_manager.cc:1398] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:20.165325 2634 tablet_bootstrap.cc:492] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:20.166249 2634 tablet_bootstrap.cc:654] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.167191 387 tablet_service.cc:1467] Processing CreateTablet for tablet d10d62c409d542ed8f80b952ccc48826 (DEFAULT_TABLE table=rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.167461 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d10d62c409d542ed8f80b952ccc48826. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.170221 2636 tablet_bootstrap.cc:492] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:20.170336 2636 ts_tablet_manager.cc:1367] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:00:20.170672 2634 tablet_bootstrap.cc:492] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:20.170684 2636 raft_consensus.cc:348] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.170753 2634 ts_tablet_manager.cc:1367] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:20.170790 2636 raft_consensus.cc:374] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.170840 2636 raft_consensus.cc:729] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.170979 2636 consensus_queue.cc:260] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.171149 2634 raft_consensus.cc:348] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:20.171241 2636 ts_tablet_manager.cc:1398] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:20.171281 2634 raft_consensus.cc:374] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.171345 2634 raft_consensus.cc:729] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.171432 2636 tablet_bootstrap.cc:492] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:20.171486 2634 consensus_queue.cc:260] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:20.171751 2634 ts_tablet_manager.cc:1398] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:20.172329 2636 tablet_bootstrap.cc:654] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.174144 2636 tablet_bootstrap.cc:492] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:20.174284 2636 ts_tablet_manager.cc:1367] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:20.174928 2636 raft_consensus.cc:348] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:20.175041 2636 raft_consensus.cc:374] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.175065 2636 raft_consensus.cc:729] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.175189 2636 consensus_queue.cc:260] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:20.175446 2636 ts_tablet_manager.cc:1398] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:20.175665 2636 tablet_bootstrap.cc:492] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:20.176573 2636 tablet_bootstrap.cc:654] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.177675 2636 tablet_bootstrap.cc:492] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:20.177752 2636 ts_tablet_manager.cc:1367] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 14:00:20.178174 2636 raft_consensus.cc:348] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.178277 2636 raft_consensus.cc:374] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.178313 2636 raft_consensus.cc:729] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.178426 2636 consensus_queue.cc:260] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.178694 2636 ts_tablet_manager.cc:1398] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:20.182453 2472 raft_consensus.cc:482] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:20.182559 2472 raft_consensus.cc:504] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.182934 2472 leader_election.cc:288] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:20.183257 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e38b601db0c4a51aa08f1f94107331e" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:20.183264 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e38b601db0c4a51aa08f1f94107331e" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:20.183385 405 raft_consensus.cc:2454] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:20.183385 542 raft_consensus.cc:2454] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:20.183689 740 leader_election.cc:302] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:20.183888 2472 raft_consensus.cc:2781] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:20.183970 2472 raft_consensus.cc:482] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:20.184037 2472 raft_consensus.cc:3037] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.184882 2472 raft_consensus.cc:504] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.185236 2472 leader_election.cc:288] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:20.185532 2610 raft_consensus.cc:482] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:20.185535 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e38b601db0c4a51aa08f1f94107331e" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:20.185649 542 raft_consensus.cc:3037] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.185601 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e38b601db0c4a51aa08f1f94107331e" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:20.185729 405 raft_consensus.cc:3037] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.185616 2610 raft_consensus.cc:504] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.186136 2610 leader_election.cc:288] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:20.186403 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d10d62c409d542ed8f80b952ccc48826" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:20.186527 804 raft_consensus.cc:2454] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:20.186645 405 raft_consensus.cc:2454] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:20.186645 542 raft_consensus.cc:2454] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:20.186735 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d10d62c409d542ed8f80b952ccc48826" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:20.186857 541 raft_consensus.cc:2454] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:20.186928 740 leader_election.cc:302] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:20.187053 344 leader_election.cc:302] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:20.187084 2472 raft_consensus.cc:2781] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:20.187177 2472 raft_consensus.cc:686] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:20.187222 2610 raft_consensus.cc:2781] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:20.187268 2610 raft_consensus.cc:482] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:20.187304 2610 raft_consensus.cc:3037] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.187320 2472 consensus_queue.cc:237] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.188509 2610 raft_consensus.cc:504] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.188848 2610 leader_election.cc:288] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:20.189178 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d10d62c409d542ed8f80b952ccc48826" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:20.189287 804 raft_consensus.cc:3037] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.190093 804 raft_consensus.cc:2454] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:20.190367 344 leader_election.cc:302] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:20.190472 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d10d62c409d542ed8f80b952ccc48826" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:20.190547 2610 raft_consensus.cc:2781] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:20.190584 541 raft_consensus.cc:3037] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.190636 2610 raft_consensus.cc:686] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:20.190783 2610 consensus_queue.cc:237] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.190827 32734 catalog_manager.cc:5484] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:20.191524 541 raft_consensus.cc:2454] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:20.193300 32731 catalog_manager.cc:5484] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:20.317704 2610 raft_consensus.cc:482] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:20.317845 2610 raft_consensus.cc:504] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:20.318274 2610 leader_election.cc:288] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:20.318632 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db6563013284583925202aeedc631ed" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:20.318706 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db6563013284583925202aeedc631ed" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:20.318770 804 raft_consensus.cc:2454] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:20.318820 674 raft_consensus.cc:2454] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:20.319047 344 leader_election.cc:302] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:20.319254 2610 raft_consensus.cc:2781] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:20.319332 2610 raft_consensus.cc:482] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:20.319371 2610 raft_consensus.cc:3037] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.320462 2610 raft_consensus.cc:504] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:20.320780 2610 leader_election.cc:288] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:20.321084 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db6563013284583925202aeedc631ed" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:20.321195 804 raft_consensus.cc:3037] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.321198 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db6563013284583925202aeedc631ed" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:20.321313 674 raft_consensus.cc:3037] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.321991 804 raft_consensus.cc:2454] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:20.322062 674 raft_consensus.cc:2454] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:20.322273 344 leader_election.cc:302] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:20.322480 2610 raft_consensus.cc:2781] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:20.322558 2610 raft_consensus.cc:686] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:20.322667 2610 consensus_queue.cc:237] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:20.324826 32731 catalog_manager.cc:5484] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:20.470899 2696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:20.471169 2696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:20.474208 2696 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_18'
I20240502 14:00:20.570427 804 raft_consensus.cc:1261] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.571033 674 raft_consensus.cc:1261] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.571440 804 raft_consensus.cc:1261] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.571461 541 raft_consensus.cc:1261] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.571940 2610 consensus_queue.cc:1035] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:20.572093 2550 consensus_queue.cc:1035] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:20.572415 2550 consensus_queue.cc:1035] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:20.573328 405 raft_consensus.cc:1261] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.573529 2610 consensus_queue.cc:1035] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:20.573546 541 raft_consensus.cc:1261] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:20.573948 2544 consensus_queue.cc:1035] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:20.574258 2544 consensus_queue.cc:1035] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 128.861 ms
time per row: 0.0644304 ms
I20240502 14:00:20.653501 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalance_test_table_19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:20.659780 387 tablet_service.cc:1467] Processing CreateTablet for tablet 7113a6da4229422099dbd604a560fb95 (DEFAULT_TABLE table=rebalance_test_table_19 [id=43c818ed117d462dbd829b88fadb4e60]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.660158 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7113a6da4229422099dbd604a560fb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.660377 520 tablet_service.cc:1467] Processing CreateTablet for tablet 7113a6da4229422099dbd604a560fb95 (DEFAULT_TABLE table=rebalance_test_table_19 [id=43c818ed117d462dbd829b88fadb4e60]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.660509 386 tablet_service.cc:1467] Processing CreateTablet for tablet 85ad896b8838472096851682071c0a9c (DEFAULT_TABLE table=rebalance_test_table_19 [id=43c818ed117d462dbd829b88fadb4e60]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.660662 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7113a6da4229422099dbd604a560fb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.660753 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85ad896b8838472096851682071c0a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.662297 2636 tablet_bootstrap.cc:492] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:20.662883 2633 tablet_bootstrap.cc:492] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:20.663129 2636 tablet_bootstrap.cc:654] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.663815 2633 tablet_bootstrap.cc:654] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.664955 784 tablet_service.cc:1467] Processing CreateTablet for tablet 85ad896b8838472096851682071c0a9c (DEFAULT_TABLE table=rebalance_test_table_19 [id=43c818ed117d462dbd829b88fadb4e60]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.665226 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85ad896b8838472096851682071c0a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.666846 2634 tablet_bootstrap.cc:492] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:20.667027 785 tablet_service.cc:1467] Processing CreateTablet for tablet f47f72fcaaae46d68f4cdab7595095ca (DEFAULT_TABLE table=rebalance_test_table_19 [id=43c818ed117d462dbd829b88fadb4e60]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.667203 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f47f72fcaaae46d68f4cdab7595095ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.667586 2634 tablet_bootstrap.cc:654] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.668455 2633 tablet_bootstrap.cc:492] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:20.668540 2633 ts_tablet_manager.cc:1367] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:20.668938 2633 raft_consensus.cc:348] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.669041 2633 raft_consensus.cc:374] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.669071 2633 raft_consensus.cc:729] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.669174 2633 consensus_queue.cc:260] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.669409 2633 ts_tablet_manager.cc:1398] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:20.670543 652 tablet_service.cc:1467] Processing CreateTablet for tablet 7113a6da4229422099dbd604a560fb95 (DEFAULT_TABLE table=rebalance_test_table_19 [id=43c818ed117d462dbd829b88fadb4e60]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.670818 652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7113a6da4229422099dbd604a560fb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.672412 2634 tablet_bootstrap.cc:492] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:20.672499 2634 ts_tablet_manager.cc:1367] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:00:20.672896 2634 raft_consensus.cc:348] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.672987 2634 raft_consensus.cc:374] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.672986 2741 tablet_bootstrap.cc:492] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:20.673017 2634 raft_consensus.cc:729] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.673130 2634 consensus_queue.cc:260] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.673125 653 tablet_service.cc:1467] Processing CreateTablet for tablet 85ad896b8838472096851682071c0a9c (DEFAULT_TABLE table=rebalance_test_table_19 [id=43c818ed117d462dbd829b88fadb4e60]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.673332 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85ad896b8838472096851682071c0a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.673386 2634 ts_tablet_manager.cc:1398] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:20.673547 2634 tablet_bootstrap.cc:492] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:20.673899 2741 tablet_bootstrap.cc:654] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.674362 2634 tablet_bootstrap.cc:654] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.675371 654 tablet_service.cc:1467] Processing CreateTablet for tablet f47f72fcaaae46d68f4cdab7595095ca (DEFAULT_TABLE table=rebalance_test_table_19 [id=43c818ed117d462dbd829b88fadb4e60]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.675428 2634 tablet_bootstrap.cc:492] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:20.675613 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f47f72fcaaae46d68f4cdab7595095ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.675673 2634 ts_tablet_manager.cc:1367] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:20.678527 2634 raft_consensus.cc:348] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.678642 2634 raft_consensus.cc:374] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.678674 2634 raft_consensus.cc:729] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.680368 520 tablet_service.cc:1467] Processing CreateTablet for tablet f47f72fcaaae46d68f4cdab7595095ca (DEFAULT_TABLE table=rebalance_test_table_19 [id=43c818ed117d462dbd829b88fadb4e60]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:20.680652 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f47f72fcaaae46d68f4cdab7595095ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:20.681301 2634 consensus_queue.cc:260] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.681634 2634 ts_tablet_manager.cc:1398] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:00:20.682766 2633 tablet_bootstrap.cc:492] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:20.683622 2636 tablet_bootstrap.cc:492] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:20.683707 2633 tablet_bootstrap.cc:654] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.683738 2636 ts_tablet_manager.cc:1367] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
I20240502 14:00:20.684288 2636 raft_consensus.cc:348] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.684401 2636 raft_consensus.cc:374] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.684437 2636 raft_consensus.cc:729] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.684578 2636 consensus_queue.cc:260] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.684900 2636 ts_tablet_manager.cc:1398] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:20.685442 2633 tablet_bootstrap.cc:492] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:20.685524 2633 ts_tablet_manager.cc:1367] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:20.685925 2633 raft_consensus.cc:348] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.686021 2633 raft_consensus.cc:374] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.686056 2633 raft_consensus.cc:729] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.686195 2633 consensus_queue.cc:260] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.686430 2633 ts_tablet_manager.cc:1398] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:20.686718 2636 tablet_bootstrap.cc:492] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:20.687563 2636 tablet_bootstrap.cc:654] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.688560 2636 tablet_bootstrap.cc:492] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:20.688645 2636 ts_tablet_manager.cc:1367] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:20.689044 2636 raft_consensus.cc:348] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.689142 2636 raft_consensus.cc:374] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.689177 2636 raft_consensus.cc:729] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.689285 2636 consensus_queue.cc:260] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.689551 2636 ts_tablet_manager.cc:1398] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:20.689990 2741 tablet_bootstrap.cc:492] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:20.690090 2741 ts_tablet_manager.cc:1367] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240502 14:00:20.690858 2741 raft_consensus.cc:348] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.690984 2741 raft_consensus.cc:374] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.691021 2741 raft_consensus.cc:729] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.691159 2741 consensus_queue.cc:260] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.691435 2741 ts_tablet_manager.cc:1398] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:20.691704 2741 tablet_bootstrap.cc:492] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:20.692754 2741 tablet_bootstrap.cc:654] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.694451 2741 tablet_bootstrap.cc:492] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:20.694561 2741 ts_tablet_manager.cc:1367] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:20.695020 2741 raft_consensus.cc:348] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.695416 2741 raft_consensus.cc:374] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.695480 2741 raft_consensus.cc:729] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.695595 2741 consensus_queue.cc:260] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.695937 2741 ts_tablet_manager.cc:1398] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:20.696128 2741 tablet_bootstrap.cc:492] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:20.697160 2741 tablet_bootstrap.cc:654] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:20.698366 2741 tablet_bootstrap.cc:492] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:20.698668 2741 ts_tablet_manager.cc:1367] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:20.699021 2741 raft_consensus.cc:348] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.699108 2741 raft_consensus.cc:374] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:20.699141 2741 raft_consensus.cc:729] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:20.699250 2741 consensus_queue.cc:260] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.699545 2741 ts_tablet_manager.cc:1398] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:20.829488 2728 raft_consensus.cc:482] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:20.829619 2728 raft_consensus.cc:504] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.830075 2728 leader_election.cc:288] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:20.830516 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad896b8838472096851682071c0a9c" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:20.830582 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad896b8838472096851682071c0a9c" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:20.830662 674 raft_consensus.cc:2454] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:20.830746 405 raft_consensus.cc:2454] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:20.830999 738 leader_election.cc:302] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:20.831156 2728 raft_consensus.cc:2781] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:20.831218 2728 raft_consensus.cc:482] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:20.831259 2728 raft_consensus.cc:3037] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.832580 2728 raft_consensus.cc:504] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.832911 2728 leader_election.cc:288] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:20.833176 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad896b8838472096851682071c0a9c" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:20.833223 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad896b8838472096851682071c0a9c" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:20.833317 674 raft_consensus.cc:3037] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.833325 405 raft_consensus.cc:3037] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.834074 405 raft_consensus.cc:2454] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:20.834076 674 raft_consensus.cc:2454] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:20.834380 738 leader_election.cc:302] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:20.834534 2728 raft_consensus.cc:2781] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:20.834620 2728 raft_consensus.cc:686] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:20.834733 2728 consensus_queue.cc:237] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:20.836870 32732 catalog_manager.cc:5484] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:20.845588 2044 raft_consensus.cc:482] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:20.845708 2044 raft_consensus.cc:504] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.846114 2044 leader_election.cc:288] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:20.846427 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f47f72fcaaae46d68f4cdab7595095ca" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:20.846565 542 raft_consensus.cc:2454] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:20.846828 606 leader_election.cc:302] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:20.846992 2044 raft_consensus.cc:2781] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:20.846957 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f47f72fcaaae46d68f4cdab7595095ca" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:20.847043 2044 raft_consensus.cc:482] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:20.847077 2044 raft_consensus.cc:3037] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.847091 806 raft_consensus.cc:2454] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:20.848114 2044 raft_consensus.cc:504] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.848438 2044 leader_election.cc:288] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:20.848713 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f47f72fcaaae46d68f4cdab7595095ca" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:20.848841 542 raft_consensus.cc:3037] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.848933 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f47f72fcaaae46d68f4cdab7595095ca" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:20.849057 806 raft_consensus.cc:3037] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.849763 542 raft_consensus.cc:2454] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:20.849812 806 raft_consensus.cc:2454] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:20.850062 606 leader_election.cc:302] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:20.850206 2044 raft_consensus.cc:2781] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:20.850319 2044 raft_consensus.cc:686] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:20.850461 2044 consensus_queue.cc:237] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.853220 32732 catalog_manager.cc:5484] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:20.885237 2606 raft_consensus.cc:482] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:20.885377 2606 raft_consensus.cc:504] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.885782 2606 leader_election.cc:288] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:20.886220 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7113a6da4229422099dbd604a560fb95" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:20.886220 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7113a6da4229422099dbd604a560fb95" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:20.886373 405 raft_consensus.cc:2454] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:20.886373 542 raft_consensus.cc:2454] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:20.886687 608 leader_election.cc:302] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:20.886869 2606 raft_consensus.cc:2781] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:20.886936 2606 raft_consensus.cc:482] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:20.886991 2606 raft_consensus.cc:3037] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.887907 2606 raft_consensus.cc:504] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.888588 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7113a6da4229422099dbd604a560fb95" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:20.888718 542 raft_consensus.cc:3037] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.888592 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7113a6da4229422099dbd604a560fb95" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:20.889022 405 raft_consensus.cc:3037] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:20.889678 542 raft_consensus.cc:2454] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:20.889907 2606 leader_election.cc:288] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:20.889936 606 leader_election.cc:302] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:20.889990 405 raft_consensus.cc:2454] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:20.890125 2606 raft_consensus.cc:2781] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:20.890209 2606 raft_consensus.cc:686] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:20.890350 2606 consensus_queue.cc:237] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:20.892367 32732 catalog_manager.cc:5484] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
W20240502 14:00:21.022785 2751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:21.023136 2751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:21.026646 2751 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_19'
I20240502 14:00:21.120239 405 raft_consensus.cc:1261] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:21.121403 2728 consensus_queue.cc:1035] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:21.122555 674 raft_consensus.cc:1261] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:21.123587 2728 consensus_queue.cc:1035] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:21.125821 542 raft_consensus.cc:1261] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:21.126111 804 raft_consensus.cc:1261] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:21.126258 2044 consensus_queue.cc:1035] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:21.127220 2736 mvcc.cc:205] Tried to move back new op lower bound from 7023240892904071168 to 7023240891740573696. Current Snapshot: MvccSnapshot[applied={T|T < 7023240892904071168}]
I20240502 14:00:21.131513 405 raft_consensus.cc:1261] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:21.132045 2606 consensus_queue.cc:1035] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:21.132795 541 raft_consensus.cc:1261] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:21.133170 2606 consensus_queue.cc:1035] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:21.135833 2044 consensus_queue.cc:1035] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:21.149588 2781 mvcc.cc:205] Tried to move back new op lower bound from 7023240892956213248 to 7023240891968462848. Current Snapshot: MvccSnapshot[applied={T|T < 7023240892941135872}]
INSERT report
rows total: 2000
time total: 122.303 ms
time per row: 0.0611515 ms
I20240502 14:00:21.194756 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.133:34751
--local_ip_for_outbound_sockets=127.26.25.133
--tserver_master_addrs=127.26.25.190:42179
--webserver_port=33117
--webserver_interface=127.26.25.133
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:00:21.312999 2803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:21.313246 2803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:21.313323 2803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:21.316567 2803 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:21.316632 2803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:21.316728 2803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.133
I20240502 14:00:21.320681 2803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.133:34751
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.25.133
--webserver_port=33117
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.2803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.133
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:21.321602 2803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:21.322721 2803 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:21.331619 2809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:21.332360 2811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:21.333170 2803 server_base.cc:1034] running on GCE node
W20240502 14:00:21.333671 2808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:21.334038 2803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:21.334591 2803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:21.335753 2803 hybrid_clock.cc:648] HybridClock initialized: now 1714658421335738 us; error 28 us; skew 500 ppm
I20240502 14:00:21.337635 2803 webserver.cc:460] Webserver started at http://127.26.25.133:33117/ using document root <none> and password file <none>
I20240502 14:00:21.338186 2803 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:21.338265 2803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:21.341624 2803 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:00:21.343468 2818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:21.344528 2803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:21.344640 2803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "4bcd7712aa844d98977a7d7e605c31f5"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:21.345034 2803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:21.368319 2803 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:21.368942 2803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:21.369099 2803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:21.369655 2803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:21.370548 2803 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:21.370597 2803 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:21.370637 2803 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:21.370659 2803 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:21.379525 2803 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.133:34751
I20240502 14:00:21.379566 2931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.133:34751 every 8 connection(s)
I20240502 14:00:21.380642 2803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 14:00:21.381242 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 2803
I20240502 14:00:21.381698 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.134:39061
--local_ip_for_outbound_sockets=127.26.25.134
--tserver_master_addrs=127.26.25.190:42179
--webserver_port=40103
--webserver_interface=127.26.25.134
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:21.394640 2932 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:21.394889 2932 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:21.395485 2932 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:21.396704 32733 ts_manager.cc:194] Re-registered known tserver with Master: 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:21.397514 32733 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.133:49595
W20240502 14:00:21.501075 2934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:21.501312 2934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:21.501355 2934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:21.504464 2934 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:21.504535 2934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:21.504606 2934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.134
I20240502 14:00:21.508790 2934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.134:39061
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.26.25.134
--webserver_port=40103
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.2934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.134
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:21.509864 2934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:21.511011 2934 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:21.520035 2941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:21.520718 2943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:21.521394 2940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:21.522082 2934 server_base.cc:1034] running on GCE node
I20240502 14:00:21.522502 2934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:21.523092 2934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:21.524240 2934 hybrid_clock.cc:648] HybridClock initialized: now 1714658421524225 us; error 31 us; skew 500 ppm
I20240502 14:00:21.526393 2934 webserver.cc:460] Webserver started at http://127.26.25.134:40103/ using document root <none> and password file <none>
I20240502 14:00:21.527192 2934 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:21.527271 2934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:21.531289 2934 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 14:00:21.533190 2950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:21.534173 2934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:21.534292 2934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "452ecbc085e54db49c3b01c20a96f860"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:21.534698 2934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:21.566147 2934 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:21.566859 2934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:21.567056 2934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:21.567648 2934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:21.568554 2934 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:21.568603 2934 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:21.568641 2934 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:21.568663 2934 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:21.577989 2934 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.134:39061
I20240502 14:00:21.578014 3064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.134:39061 every 8 connection(s)
I20240502 14:00:21.579066 2934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 14:00:21.580726 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 2934
I20240502 14:00:21.581156 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.135:45509
--local_ip_for_outbound_sockets=127.26.25.135
--tserver_master_addrs=127.26.25.190:42179
--webserver_port=37277
--webserver_interface=127.26.25.135
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:21.589357 3065 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:21.589599 3065 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:21.590276 3065 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:21.591392 32733 ts_manager.cc:194] Re-registered known tserver with Master: 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:21.592017 32733 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.134:50281
W20240502 14:00:21.708621 3068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:21.708876 3068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:21.708953 3068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:21.712116 3068 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:21.712203 3068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:21.712301 3068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.135
I20240502 14:00:21.716540 3068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.135:45509
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.26.25.135
--webserver_port=37277
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.3068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.135
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:21.717465 3068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:21.718596 3068 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:21.727605 3075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:21.728120 3077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:21.729130 3074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:21.729882 3068 server_base.cc:1034] running on GCE node
I20240502 14:00:21.730329 3068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:21.730919 3068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:21.732089 3068 hybrid_clock.cc:648] HybridClock initialized: now 1714658421732068 us; error 44 us; skew 500 ppm
I20240502 14:00:21.734174 3068 webserver.cc:460] Webserver started at http://127.26.25.135:37277/ using document root <none> and password file <none>
I20240502 14:00:21.734818 3068 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:21.734920 3068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:21.738723 3068 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:21.741029 3083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:21.742069 3068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:21.742183 3068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "2b6796cf0e2e496e9dd641beb432f485"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:21.742604 3068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:21.787030 3068 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:21.787708 3068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:21.787904 3068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:21.788462 3068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:21.789359 3068 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:21.789405 3068 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:21.789451 3068 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:21.789484 3068 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:21.798378 3068 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.135:45509
I20240502 14:00:21.798409 3197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.135:45509 every 8 connection(s)
I20240502 14:00:21.799502 3068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 14:00:21.807978 3198 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:21.808202 3198 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:21.808415 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 3068
I20240502 14:00:21.808871 3198 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:21.808878 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.136:43131
--local_ip_for_outbound_sockets=127.26.25.136
--tserver_master_addrs=127.26.25.190:42179
--webserver_port=38065
--webserver_interface=127.26.25.136
--builtin_ntp_servers=127.26.25.148:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:21.810007 32733 ts_manager.cc:194] Re-registered known tserver with Master: 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:21.810680 32733 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.135:52093
W20240502 14:00:21.930292 3201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:21.930516 3201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:21.930577 3201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:21.933917 3201 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:21.933986 3201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:21.934082 3201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.136
I20240502 14:00:21.937796 3201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.136:43131
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.26.25.136
--webserver_port=38065
--tserver_master_addrs=127.26.25.190:42179
--never_fsync=true
--heap_profile_path=/tmp/kudu.3201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.136
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:21.938808 3201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:21.940161 3201 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:21.949079 3207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:21.949573 3206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:21.950501 3209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:21.951293 3201 server_base.cc:1034] running on GCE node
I20240502 14:00:21.951668 3201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:21.952260 3201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:21.953402 3201 hybrid_clock.cc:648] HybridClock initialized: now 1714658421953376 us; error 40 us; skew 500 ppm
I20240502 14:00:21.964646 3201 webserver.cc:460] Webserver started at http://127.26.25.136:38065/ using document root <none> and password file <none>
I20240502 14:00:21.965263 3201 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:21.965345 3201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:21.968670 3201 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:00:21.970506 3215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:21.971444 3201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:21.971544 3201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "da3f8160557e454f8547f45e80c2f013"
format_stamp: "Formatted at 2024-05-02 14:00:05 on dist-test-slave-sldb"
I20240502 14:00:21.972005 3201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:21.996591 3201 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:21.997478 3201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:21.997689 3201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:21.998322 3201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:21.999564 3201 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:21.999615 3201 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:21.999679 3201 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:21.999706 3201 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:22.011202 3201 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.136:43131
I20240502 14:00:22.011224 3329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.136:43131 every 8 connection(s)
I20240502 14:00:22.012370 3201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 14:00:22.015570 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 3201
I20240502 14:00:22.019768 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:22.026048 3330 heartbeater.cc:347] Connected to a master server at 127.26.25.190:42179
I20240502 14:00:22.026295 3330 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:22.026885 3330 heartbeater.cc:511] Master 127.26.25.190:42179 requested a full tablet report, sending...
I20240502 14:00:22.028053 32732 ts_manager.cc:194] Re-registered known tserver with Master: da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:22.028920 32732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.136:33019
I20240502 14:00:22.048573 520 tablet_service.cc:1467] Processing CreateTablet for tablet 9889f98262f84780bfe4ceba3a5a7d81 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=924ebfa03ca64a8d93c3964905d233c3]), partition=
I20240502 14:00:22.048882 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9889f98262f84780bfe4ceba3a5a7d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.050762 3346 tablet_bootstrap.cc:492] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:22.051682 3346 tablet_bootstrap.cc:654] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.052925 3346 tablet_bootstrap.cc:492] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:22.053112 3346 ts_tablet_manager.cc:1367] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:22.053694 3346 raft_consensus.cc:348] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:22.057339 3346 raft_consensus.cc:374] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.057430 3346 raft_consensus.cc:729] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.057677 3346 consensus_queue.cc:260] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:22.059311 3346 ts_tablet_manager.cc:1398] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:00:22.059348 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 9889f98262f84780bfe4ceba3a5a7d81 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=924ebfa03ca64a8d93c3964905d233c3]), partition=
I20240502 14:00:22.060106 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9889f98262f84780bfe4ceba3a5a7d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.065621 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 9889f98262f84780bfe4ceba3a5a7d81 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=924ebfa03ca64a8d93c3964905d233c3]), partition=
I20240502 14:00:22.066352 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9889f98262f84780bfe4ceba3a5a7d81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.076316 3348 tablet_bootstrap.cc:492] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:22.077339 3349 tablet_bootstrap.cc:492] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:22.078837 3348 tablet_bootstrap.cc:654] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.079756 3348 log.cc:824] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:22.080590 3349 tablet_bootstrap.cc:654] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.081276 3349 log.cc:824] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:22.082854 3349 tablet_bootstrap.cc:492] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:22.083052 3349 ts_tablet_manager.cc:1367] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 14:00:22.084920 3348 tablet_bootstrap.cc:492] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:22.085107 3348 ts_tablet_manager.cc:1367] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20240502 14:00:22.086079 3349 raft_consensus.cc:348] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:22.086257 3349 raft_consensus.cc:374] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.086341 3349 raft_consensus.cc:729] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.086738 3349 consensus_queue.cc:260] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:22.087355 3198 heartbeater.cc:503] Master 127.26.25.190:42179 was elected leader, sending a full tablet report...
I20240502 14:00:22.087571 3349 ts_tablet_manager.cc:1398] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 14:00:22.089102 3348 raft_consensus.cc:348] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:22.089356 3348 raft_consensus.cc:374] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.089421 3348 raft_consensus.cc:729] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.089883 3348 consensus_queue.cc:260] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:22.090643 3065 heartbeater.cc:503] Master 127.26.25.190:42179 was elected leader, sending a full tablet report...
I20240502 14:00:22.090704 3348 ts_tablet_manager.cc:1398] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20240502 14:00:22.173316 3353 raft_consensus.cc:482] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:22.173506 3353 raft_consensus.cc:504] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:22.174513 3353 leader_election.cc:288] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:22.180094 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9889f98262f84780bfe4ceba3a5a7d81" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:22.180254 541 raft_consensus.cc:2454] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:22.180214 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9889f98262f84780bfe4ceba3a5a7d81" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:22.180481 3152 raft_consensus.cc:2454] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:22.180665 2951 leader_election.cc:302] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:22.181169 3353 raft_consensus.cc:2781] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:22.181246 3353 raft_consensus.cc:482] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:22.181299 3353 raft_consensus.cc:3037] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.182410 3353 raft_consensus.cc:504] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:22.182703 3353 leader_election.cc:288] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:22.183023 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9889f98262f84780bfe4ceba3a5a7d81" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:22.183074 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9889f98262f84780bfe4ceba3a5a7d81" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:22.183127 3152 raft_consensus.cc:3037] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.183188 541 raft_consensus.cc:3037] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.184010 541 raft_consensus.cc:2454] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:22.184010 3152 raft_consensus.cc:2454] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:22.184263 2951 leader_election.cc:302] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:22.184481 3353 raft_consensus.cc:2781] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:22.184703 3353 raft_consensus.cc:686] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:22.184999 3353 consensus_queue.cc:237] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:22.188167 32733 catalog_manager.cc:5484] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:22.295365 3132 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:22.299787 2999 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20240502 14:00:22.300717 3199 tablet.cc:2369] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:22.302925 32725 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:22.307750 3264 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:22.311885 654 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240502 14:00:22.313159 520 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 14:00:22.314754 2866 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240502 14:00:22.330197 3066 tablet.cc:2369] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:22.332463 785 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 14:00:22.342990 3152 raft_consensus.cc:1261] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:22.343713 3353 consensus_queue.cc:1035] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:22.346884 386 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 14:00:22.347308 540 raft_consensus.cc:1261] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:22.347987 3353 consensus_queue.cc:1035] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:22.348450 32732 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 9889f98262f84780bfe4ceba3a5a7d81 (table rebalancer_dynamic_table [id=924ebfa03ca64a8d93c3964905d233c3]). Expected version 1 got 0
I20240502 14:00:22.354293 32725 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet 9889f98262f84780bfe4ceba3a5a7d81 (table rebalancer_dynamic_table [id=924ebfa03ca64a8d93c3964905d233c3]). Expected version 1 got 0
I20240502 14:00:22.354552 3416 mvcc.cc:205] Tried to move back new op lower bound from 7023240897910022144 to 7023240897272471552. Current Snapshot: MvccSnapshot[applied={T|T < 7023240897910022144}]
I20240502 14:00:22.355502 32727 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 9889f98262f84780bfe4ceba3a5a7d81 (table rebalancer_dynamic_table [id=924ebfa03ca64a8d93c3964905d233c3]). Expected version 1 got 0
I20240502 14:00:22.355996 3422 tablet.cc:1722] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:22.357365 3422 tablet.cc:1620] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:22.361776 32732 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 9889f98262f84780bfe4ceba3a5a7d81 (table rebalancer_dynamic_table [id=924ebfa03ca64a8d93c3964905d233c3]). Expected version 1 got 0
I20240502 14:00:22.362277 32727 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=924ebfa03ca64a8d93c3964905d233c3] alter complete (version 1)
I20240502 14:00:22.367091 32732 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 9889f98262f84780bfe4ceba3a5a7d81 (table rebalancer_dynamic_table [id=924ebfa03ca64a8d93c3964905d233c3]). Expected version 1 got 0
I20240502 14:00:22.374822 3418 mvcc.cc:205] Tried to move back new op lower bound from 7023240897910022144 to 7023240897272471552. Current Snapshot: MvccSnapshot[applied={T|T < 7023240897910022144}]
I20240502 14:00:22.382428 3425 tablet.cc:1722] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:22.382479 3424 tablet.cc:1722] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:22.383147 3424 tablet.cc:1620] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:22.383145 3425 tablet.cc:1620] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:22.399973 2932 heartbeater.cc:503] Master 127.26.25.190:42179 was elected leader, sending a full tablet report...
I20240502 14:00:22.465034 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:22.472915 3422 tablet.cc:1722] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:22.473248 3422 tablet.cc:1620] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:22.478343 3424 tablet.cc:1722] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:22.478689 3424 tablet.cc:1620] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:22.480111 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=924ebfa03ca64a8d93c3964905d233c3] alter complete (version 2)
I20240502 14:00:22.482695 3425 tablet.cc:1722] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:22.483186 3425 tablet.cc:1620] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:22.519780 32733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_0" } modify_external_catalogs: true
I20240502 14:00:22.520022 32733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_0" } modify_external_catalogs: true
I20240502 14:00:22.520188 32727 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:22.524616 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 7ac54cbde3184904a66f2383cbc3ca8a
I20240502 14:00:22.525027 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 50fbf855074e4c5c89064df2bbf59c06
I20240502 14:00:22.525223 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 0b5c9e82af1e4db192609694441e3705
I20240502 14:00:22.526372 654 tablet_service.cc:1514] Processing DeleteTablet for tablet 50fbf855074e4c5c89064df2bbf59c06 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:22.526532 388 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b5c9e82af1e4db192609694441e3705 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:22.526777 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b5c9e82af1e4db192609694441e3705 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:22.526978 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ac54cbde3184904a66f2383cbc3ca8a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:22.527333 522 tablet_service.cc:1514] Processing DeleteTablet for tablet 50fbf855074e4c5c89064df2bbf59c06 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:22.527468 3485 tablet_replica.cc:330] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:22.527518 386 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ac54cbde3184904a66f2383cbc3ca8a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:22.527643 387 tablet_service.cc:1514] Processing DeleteTablet for tablet 50fbf855074e4c5c89064df2bbf59c06 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:22.527805 3485 raft_consensus.cc:2229] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:22.527841 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ac54cbde3184904a66f2383cbc3ca8a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:22.528038 3485 raft_consensus.cc:2258] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.528148 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b5c9e82af1e4db192609694441e3705 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:22.528396 2866 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:22.529568 3485 ts_tablet_manager.cc:1875] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.531141 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb2f6c66b714668ab1b698fa4f6cc6c (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=c9979d0690074e1484d971eed95aad38]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:22.531530 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb2f6c66b714668ab1b698fa4f6cc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.532312 3485 ts_tablet_manager.cc:1888] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:22.532476 3485 log.cc:1196] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/0b5c9e82af1e4db192609694441e3705
I20240502 14:00:22.532591 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb2f6c66b714668ab1b698fa4f6cc6c (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=c9979d0690074e1484d971eed95aad38]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:22.532775 3485 ts_tablet_manager.cc:1909] T 0b5c9e82af1e4db192609694441e3705 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:22.532940 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb2f6c66b714668ab1b698fa4f6cc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.533825 3348 tablet_bootstrap.cc:492] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:22.534384 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 5e922f7cee8046e7b4263dd57b8d1185 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=c9979d0690074e1484d971eed95aad38]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:22.534612 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e922f7cee8046e7b4263dd57b8d1185. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.534653 3348 tablet_bootstrap.cc:654] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.535171 3485 tablet_replica.cc:330] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:22.535292 3485 raft_consensus.cc:2229] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:22.535504 3349 tablet_bootstrap.cc:492] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:22.536406 3349 tablet_bootstrap.cc:654] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.536456 3485 raft_consensus.cc:2258] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.537017 653 tablet_service.cc:1467] Processing CreateTablet for tablet 9004b8cfea064786a1522e7ced149167 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=c9979d0690074e1484d971eed95aad38]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:22.537286 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9004b8cfea064786a1522e7ced149167. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.538173 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 0b5c9e82af1e4db192609694441e3705 (table rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]) successfully deleted
I20240502 14:00:22.538517 387 tablet_service.cc:1467] Processing CreateTablet for tablet 9004b8cfea064786a1522e7ced149167 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=c9979d0690074e1484d971eed95aad38]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:22.538833 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9004b8cfea064786a1522e7ced149167. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.540037 3485 ts_tablet_manager.cc:1875] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.540855 2999 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:22.542385 3485 ts_tablet_manager.cc:1888] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:22.542447 3485 log.cc:1196] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/7ac54cbde3184904a66f2383cbc3ca8a
I20240502 14:00:22.542696 3485 ts_tablet_manager.cc:1909] T 7ac54cbde3184904a66f2383cbc3ca8a P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:22.543241 3485 tablet_replica.cc:330] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:22.543421 3485 raft_consensus.cc:2229] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:22.543684 3485 raft_consensus.cc:2258] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.544222 3485 ts_tablet_manager.cc:1875] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.545250 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 7ac54cbde3184904a66f2383cbc3ca8a (table rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]) successfully deleted
I20240502 14:00:22.548837 3132 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:22.549774 653 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240502 14:00:22.550115 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 5e922f7cee8046e7b4263dd57b8d1185 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=c9979d0690074e1484d971eed95aad38]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:22.551100 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e922f7cee8046e7b4263dd57b8d1185. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.557375 3264 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:22.562647 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb2f6c66b714668ab1b698fa4f6cc6c (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=c9979d0690074e1484d971eed95aad38]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:22.563328 3494 tablet_bootstrap.cc:492] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:22.563603 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb2f6c66b714668ab1b698fa4f6cc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.564133 3262 tablet_service.cc:1467] Processing CreateTablet for tablet 5e922f7cee8046e7b4263dd57b8d1185 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=c9979d0690074e1484d971eed95aad38]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:22.564289 3495 tablet_bootstrap.cc:492] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:22.565078 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 9004b8cfea064786a1522e7ced149167 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=c9979d0690074e1484d971eed95aad38]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:22.566507 3495 tablet_bootstrap.cc:654] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.566752 3486 tablet_replica.cc:330] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:22.567008 3488 tablet_replica.cc:330] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:22.567343 522 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 14:00:22.567651 3262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e922f7cee8046e7b4263dd57b8d1185. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.567960 3486 raft_consensus.cc:2229] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:22.568266 3486 raft_consensus.cc:2258] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.569115 3486 ts_tablet_manager.cc:1875] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.571344 3486 ts_tablet_manager.cc:1888] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:22.571419 3486 log.cc:1196] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/7ac54cbde3184904a66f2383cbc3ca8a
I20240502 14:00:22.571719 3486 ts_tablet_manager.cc:1909] T 7ac54cbde3184904a66f2383cbc3ca8a P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:22.572366 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9004b8cfea064786a1522e7ced149167. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.573236 3349 tablet_bootstrap.cc:492] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:22.573328 3349 ts_tablet_manager.cc:1367] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.000s
I20240502 14:00:22.573997 3349 raft_consensus.cc:348] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.574131 3349 raft_consensus.cc:374] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.574172 3349 raft_consensus.cc:729] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.574288 3349 consensus_queue.cc:260] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.574563 3349 ts_tablet_manager.cc:1398] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:22.574924 3484 tablet_replica.cc:330] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:22.575191 3488 raft_consensus.cc:2229] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:22.576354 3484 raft_consensus.cc:2229] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:22.576566 3488 raft_consensus.cc:2258] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.577113 3484 raft_consensus.cc:2258] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.578155 3494 tablet_bootstrap.cc:654] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.579222 3484 ts_tablet_manager.cc:1875] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.580457 3488 ts_tablet_manager.cc:1875] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.582149 3348 tablet_bootstrap.cc:492] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:22.582250 3348 ts_tablet_manager.cc:1367] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.048s user 0.001s sys 0.001s
I20240502 14:00:22.582867 3348 raft_consensus.cc:348] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.582993 3348 raft_consensus.cc:374] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.583030 3348 raft_consensus.cc:729] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.583159 3348 consensus_queue.cc:260] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.583402 3348 ts_tablet_manager.cc:1398] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 14:00:22.583592 3348 tablet_bootstrap.cc:492] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:22.584583 3348 tablet_bootstrap.cc:654] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.585295 387 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 14:00:22.583626 3493 tablet_bootstrap.cc:492] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:22.588708 3493 tablet_bootstrap.cc:654] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.589649 3495 log.cc:824] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:22.593484 3488 ts_tablet_manager.cc:1888] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:22.593425 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:22.593696 3488 log.cc:1196] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/7ac54cbde3184904a66f2383cbc3ca8a
I20240502 14:00:22.594060 3488 ts_tablet_manager.cc:1909] T 7ac54cbde3184904a66f2383cbc3ca8a P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:22.595310 3488 tablet_replica.cc:330] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:22.595307 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 7ac54cbde3184904a66f2383cbc3ca8a (table rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]) successfully deleted
I20240502 14:00:22.595530 3488 raft_consensus.cc:2229] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:22.595872 3488 raft_consensus.cc:2258] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.596426 3488 ts_tablet_manager.cc:1875] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.598424 3485 ts_tablet_manager.cc:1888] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:22.598513 3485 log.cc:1196] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/50fbf855074e4c5c89064df2bbf59c06
I20240502 14:00:22.598775 3485 ts_tablet_manager.cc:1909] T 50fbf855074e4c5c89064df2bbf59c06 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:22.599813 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 7ac54cbde3184904a66f2383cbc3ca8a (table rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]) successfully deleted
I20240502 14:00:22.600173 3486 tablet_replica.cc:330] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:22.600378 3486 raft_consensus.cc:2229] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:22.600575 3486 raft_consensus.cc:2258] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.601158 3486 ts_tablet_manager.cc:1875] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.602661 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 50fbf855074e4c5c89064df2bbf59c06 (table rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]) successfully deleted
I20240502 14:00:22.603111 3484 ts_tablet_manager.cc:1888] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:22.603260 3484 log.cc:1196] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/50fbf855074e4c5c89064df2bbf59c06
I20240502 14:00:22.603336 3422 tablet.cc:1722] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:22.603641 3484 ts_tablet_manager.cc:1909] T 50fbf855074e4c5c89064df2bbf59c06 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:22.605594 3495 tablet_bootstrap.cc:492] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:22.606142 3494 tablet_bootstrap.cc:492] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:22.606268 3494 ts_tablet_manager.cc:1367] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.000s
I20240502 14:00:22.606737 3494 raft_consensus.cc:348] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.606890 3494 raft_consensus.cc:374] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.606947 3494 raft_consensus.cc:729] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.607143 3494 consensus_queue.cc:260] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.607595 3494 ts_tablet_manager.cc:1398] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:22.607931 3495 ts_tablet_manager.cc:1367] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.044s user 0.005s sys 0.000s
I20240502 14:00:22.609879 3424 tablet.cc:1722] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:22.610422 3484 tablet_replica.cc:330] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:22.610478 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 50fbf855074e4c5c89064df2bbf59c06 (table rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]) successfully deleted
I20240502 14:00:22.611155 3493 tablet_bootstrap.cc:492] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:22.612622 3484 raft_consensus.cc:2229] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:22.613863 3484 raft_consensus.cc:2258] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.614580 3484 ts_tablet_manager.cc:1875] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.612793 3493 ts_tablet_manager.cc:1367] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20240502 14:00:22.615142 3493 raft_consensus.cc:348] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.615332 3493 raft_consensus.cc:374] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.615415 3493 raft_consensus.cc:729] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.615629 3493 consensus_queue.cc:260] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.616052 3493 ts_tablet_manager.cc:1398] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:22.623283 784 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 14:00:22.623360 32727 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=924ebfa03ca64a8d93c3964905d233c3] alter complete (version 3)
I20240502 14:00:22.624194 3497 tablet_bootstrap.cc:492] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:22.624917 3488 ts_tablet_manager.cc:1888] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:22.625504 3348 tablet_bootstrap.cc:492] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:22.625592 3348 ts_tablet_manager.cc:1367] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.000s
I20240502 14:00:22.626005 3348 raft_consensus.cc:348] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.626111 3348 raft_consensus.cc:374] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.626137 3348 raft_consensus.cc:729] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.626240 3348 consensus_queue.cc:260] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.626478 3348 ts_tablet_manager.cc:1398] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:22.628686 3495 raft_consensus.cc:348] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.629864 3486 ts_tablet_manager.cc:1888] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:22.629997 3486 log.cc:1196] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/50fbf855074e4c5c89064df2bbf59c06
I20240502 14:00:22.630838 3484 ts_tablet_manager.cc:1888] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:22.631449 3497 tablet_bootstrap.cc:654] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.631930 3425 tablet.cc:1722] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:22.632335 3486 ts_tablet_manager.cc:1909] T 50fbf855074e4c5c89064df2bbf59c06 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:22.633003 3488 log.cc:1196] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/0b5c9e82af1e4db192609694441e3705
I20240502 14:00:22.633002 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 50fbf855074e4c5c89064df2bbf59c06 (table rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]) successfully deleted
I20240502 14:00:22.633327 3488 ts_tablet_manager.cc:1909] T 0b5c9e82af1e4db192609694441e3705 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:22.633441 3484 log.cc:1196] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/0b5c9e82af1e4db192609694441e3705
I20240502 14:00:22.633677 3497 log.cc:824] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:22.633803 3484 ts_tablet_manager.cc:1909] T 0b5c9e82af1e4db192609694441e3705 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:22.635952 3495 raft_consensus.cc:374] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.636083 3495 raft_consensus.cc:729] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.636059 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 0b5c9e82af1e4db192609694441e3705 (table rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]) successfully deleted
I20240502 14:00:22.636679 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 0b5c9e82af1e4db192609694441e3705 (table rebalance_test_table_0 [id=164cdbf8715646f8a28090207b722bb6]) successfully deleted
I20240502 14:00:22.637013 3495 consensus_queue.cc:260] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.638353 3495 ts_tablet_manager.cc:1398] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.030s user 0.002s sys 0.004s
I20240502 14:00:22.641093 3497 tablet_bootstrap.cc:492] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:22.641357 3497 ts_tablet_manager.cc:1367] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20240502 14:00:22.644979 3497 raft_consensus.cc:348] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.645180 3497 raft_consensus.cc:374] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.645267 3497 raft_consensus.cc:729] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.645722 3497 consensus_queue.cc:260] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.646706 3330 heartbeater.cc:503] Master 127.26.25.190:42179 was elected leader, sending a full tablet report...
I20240502 14:00:22.646749 3497 ts_tablet_manager.cc:1398] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20240502 14:00:22.647063 3497 tablet_bootstrap.cc:492] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:22.648249 3497 tablet_bootstrap.cc:654] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.652082 3503 raft_consensus.cc:482] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:22.652256 3503 raft_consensus.cc:504] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.653923 3503 leader_election.cc:288] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:22.654330 3497 tablet_bootstrap.cc:492] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:22.654464 3497 ts_tablet_manager.cc:1367] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:22.655093 3497 raft_consensus.cc:348] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.655193 3497 raft_consensus.cc:374] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.655222 3497 raft_consensus.cc:729] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.655340 3497 consensus_queue.cc:260] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.655654 3497 ts_tablet_manager.cc:1398] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:22.655908 3497 tablet_bootstrap.cc:492] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:22.657020 3497 tablet_bootstrap.cc:654] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.661041 3497 tablet_bootstrap.cc:492] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:22.661129 3497 ts_tablet_manager.cc:1367] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:22.661550 3497 raft_consensus.cc:348] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.661652 3497 raft_consensus.cc:374] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.661684 3497 raft_consensus.cc:729] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.661800 3497 consensus_queue.cc:260] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.662057 3497 ts_tablet_manager.cc:1398] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:22.662273 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e922f7cee8046e7b4263dd57b8d1185" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:22.662578 3284 raft_consensus.cc:2454] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:22.663583 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e922f7cee8046e7b4263dd57b8d1185" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:22.663794 3019 raft_consensus.cc:2454] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:22.664338 2819 leader_election.cc:302] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:22.664664 3503 raft_consensus.cc:2781] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:22.664732 3503 raft_consensus.cc:482] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:22.664774 3503 raft_consensus.cc:3037] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.665890 3503 raft_consensus.cc:504] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.666245 3503 leader_election.cc:288] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:22.666553 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e922f7cee8046e7b4263dd57b8d1185" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:22.666668 3284 raft_consensus.cc:3037] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.667112 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e922f7cee8046e7b4263dd57b8d1185" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:22.667217 3019 raft_consensus.cc:3037] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.667747 3284 raft_consensus.cc:2454] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:22.668166 2820 leader_election.cc:302] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:22.668231 3019 raft_consensus.cc:2454] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:22.668499 3503 raft_consensus.cc:2781] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:22.668720 3503 raft_consensus.cc:686] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:22.669029 3503 consensus_queue.cc:237] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.672624 32727 catalog_manager.cc:5484] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:22.727329 32727 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:22.727507 32727 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:22.731376 32727 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 9889f98262f84780bfe4ceba3a5a7d81
I20240502 14:00:22.732045 522 tablet_service.cc:1514] Processing DeleteTablet for tablet 9889f98262f84780bfe4ceba3a5a7d81 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:22.732017 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 9889f98262f84780bfe4ceba3a5a7d81 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:22.732237 3486 tablet_replica.cc:330] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:22.732373 3486 raft_consensus.cc:2229] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:22.732455 3552 tablet_replica.cc:330] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:22.732504 3486 raft_consensus.cc:2258] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.732793 3552 raft_consensus.cc:2229] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:22.733275 3552 raft_consensus.cc:2258] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.733409 3486 ts_tablet_manager.cc:1875] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.734563 3552 ts_tablet_manager.cc:1875] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.735633 3486 ts_tablet_manager.cc:1888] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:22.735715 3486 log.cc:1196] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/9889f98262f84780bfe4ceba3a5a7d81
I20240502 14:00:22.736032 3486 ts_tablet_manager.cc:1909] T 9889f98262f84780bfe4ceba3a5a7d81 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:22.738230 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 9889f98262f84780bfe4ceba3a5a7d81 (table rebalancer_dynamic_table_new_name [id=924ebfa03ca64a8d93c3964905d233c3]) successfully deleted
I20240502 14:00:22.738965 3552 ts_tablet_manager.cc:1888] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:22.739116 3552 log.cc:1196] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/9889f98262f84780bfe4ceba3a5a7d81
I20240502 14:00:22.739264 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 9889f98262f84780bfe4ceba3a5a7d81 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:22 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:22.739506 3552 ts_tablet_manager.cc:1909] T 9889f98262f84780bfe4ceba3a5a7d81 P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:22.740140 3553 tablet_replica.cc:330] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:22.740494 3553 raft_consensus.cc:2229] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:22.740782 3553 raft_consensus.cc:2258] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:22.742336 3553 ts_tablet_manager.cc:1875] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:22.743579 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 9889f98262f84780bfe4ceba3a5a7d81 (table rebalancer_dynamic_table_new_name [id=924ebfa03ca64a8d93c3964905d233c3]) successfully deleted
I20240502 14:00:22.747113 3553 ts_tablet_manager.cc:1888] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:22.747267 3553 log.cc:1196] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/9889f98262f84780bfe4ceba3a5a7d81
I20240502 14:00:22.747666 3553 ts_tablet_manager.cc:1909] T 9889f98262f84780bfe4ceba3a5a7d81 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:22.748867 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 9889f98262f84780bfe4ceba3a5a7d81 (table rebalancer_dynamic_table_new_name [id=924ebfa03ca64a8d93c3964905d233c3]) successfully deleted
W20240502 14:00:22.763908 3331 tablet.cc:2369] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:22.785780 3504 raft_consensus.cc:482] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:22.785915 3504 raft_consensus.cc:504] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.786938 3504 leader_election.cc:288] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:22.791409 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb2f6c66b714668ab1b698fa4f6cc6c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:22.791409 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb2f6c66b714668ab1b698fa4f6cc6c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:22.791558 3152 raft_consensus.cc:2454] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:22.791561 3019 raft_consensus.cc:2454] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:22.791918 3219 leader_election.cc:302] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:22.792186 3504 raft_consensus.cc:2781] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:22.792253 3504 raft_consensus.cc:482] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:22.792280 3504 raft_consensus.cc:3037] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.793115 3504 raft_consensus.cc:504] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.793452 3504 leader_election.cc:288] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:22.793767 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb2f6c66b714668ab1b698fa4f6cc6c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:22.793807 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb2f6c66b714668ab1b698fa4f6cc6c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:22.793905 3019 raft_consensus.cc:3037] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.793910 3152 raft_consensus.cc:3037] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.794823 3019 raft_consensus.cc:2454] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:22.794946 3152 raft_consensus.cc:2454] T 7eb2f6c66b714668ab1b698fa4f6cc6c P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:22.795154 3216 leader_election.cc:302] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:22.795423 3504 raft_consensus.cc:2781] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:22.795641 3504 raft_consensus.cc:686] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:22.796258 3504 consensus_queue.cc:237] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:22.799247 32731 catalog_manager.cc:5484] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:22.803997 3504 raft_consensus.cc:482] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:22.804078 3504 raft_consensus.cc:504] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.804502 3504 leader_election.cc:288] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:22.808872 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9004b8cfea064786a1522e7ced149167" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:22.809020 409 raft_consensus.cc:2454] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:22.809290 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9004b8cfea064786a1522e7ced149167" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:22.809358 3218 leader_election.cc:302] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:22.809437 672 raft_consensus.cc:2454] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:22.809530 3504 raft_consensus.cc:2781] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:22.809616 3504 raft_consensus.cc:482] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:22.809680 3504 raft_consensus.cc:3037] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.810495 3504 raft_consensus.cc:504] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.810827 3504 leader_election.cc:288] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:22.811152 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9004b8cfea064786a1522e7ced149167" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:22.811266 672 raft_consensus.cc:3037] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.811960 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9004b8cfea064786a1522e7ced149167" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:22.812076 409 raft_consensus.cc:3037] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.812237 672 raft_consensus.cc:2454] T 9004b8cfea064786a1522e7ced149167 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:22.812498 3216 leader_election.cc:302] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:22.812721 3504 raft_consensus.cc:2781] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:22.812815 3504 raft_consensus.cc:686] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:22.813102 409 raft_consensus.cc:2454] T 9004b8cfea064786a1522e7ced149167 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:22.813037 3504 consensus_queue.cc:237] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.815277 32731 catalog_manager.cc:5484] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:22.839133 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:22.844601 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 54618de152b04381ae6c792a57d0a9e6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8f94f7b869a340139441aa113c7682a1]), partition=
I20240502 14:00:22.844892 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54618de152b04381ae6c792a57d0a9e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.845008 3262 tablet_service.cc:1467] Processing CreateTablet for tablet 54618de152b04381ae6c792a57d0a9e6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8f94f7b869a340139441aa113c7682a1]), partition=
I20240502 14:00:22.845269 3262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54618de152b04381ae6c792a57d0a9e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.846923 3495 tablet_bootstrap.cc:492] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:22.847307 3497 tablet_bootstrap.cc:492] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:22.847741 3495 tablet_bootstrap.cc:654] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.848160 3497 tablet_bootstrap.cc:654] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.848745 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 54618de152b04381ae6c792a57d0a9e6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8f94f7b869a340139441aa113c7682a1]), partition=
I20240502 14:00:22.849025 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54618de152b04381ae6c792a57d0a9e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.849032 3495 tablet_bootstrap.cc:492] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:22.849102 3495 ts_tablet_manager.cc:1367] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:22.849655 3495 raft_consensus.cc:348] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.849781 3495 raft_consensus.cc:374] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.849813 3495 raft_consensus.cc:729] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.849927 3495 consensus_queue.cc:260] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.850086 3497 tablet_bootstrap.cc:492] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:22.850157 3497 ts_tablet_manager.cc:1367] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:22.850195 3495 ts_tablet_manager.cc:1398] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:22.850536 3497 raft_consensus.cc:348] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.850631 3497 raft_consensus.cc:374] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.850661 3497 raft_consensus.cc:729] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.850764 3497 consensus_queue.cc:260] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.851027 3497 ts_tablet_manager.cc:1398] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:22.852393 3349 tablet_bootstrap.cc:492] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:22.853533 3349 tablet_bootstrap.cc:654] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.855253 3349 tablet_bootstrap.cc:492] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:22.855353 3349 ts_tablet_manager.cc:1367] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:22.855788 3349 raft_consensus.cc:348] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.855923 3349 raft_consensus.cc:374] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.855948 3349 raft_consensus.cc:729] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.856067 3349 consensus_queue.cc:260] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:22.856307 3349 ts_tablet_manager.cc:1398] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 14:00:22.882537 2933 tablet.cc:2369] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:23.028201 32732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_2" } modify_external_catalogs: true
I20240502 14:00:23.028371 32732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_2" } modify_external_catalogs: true
I20240502 14:00:23.030819 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet ea8c5159a1e24471ab878ffb50a6df05
I20240502 14:00:23.031220 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet aa30d86ef587427db4213f5c178a744a
I20240502 14:00:23.031507 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 2432b3ab36a54fbfa108031f67634e2c
I20240502 14:00:23.031478 784 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8c5159a1e24471ab878ffb50a6df05 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:23.031687 522 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8c5159a1e24471ab878ffb50a6df05 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:23.031864 388 tablet_service.cc:1514] Processing DeleteTablet for tablet 2432b3ab36a54fbfa108031f67634e2c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:23.031929 3488 tablet_replica.cc:330] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:23.032064 3485 tablet_replica.cc:330] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:23.032094 3488 raft_consensus.cc:2229] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:23.032152 3486 tablet_replica.cc:330] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:23.032189 3485 raft_consensus.cc:2229] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.032284 3486 raft_consensus.cc:2229] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.032313 3485 raft_consensus.cc:2258] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.032321 3488 raft_consensus.cc:2258] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.032418 3486 raft_consensus.cc:2258] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.032830 3485 ts_tablet_manager.cc:1875] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.032855 3488 ts_tablet_manager.cc:1875] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.032879 3486 ts_tablet_manager.cc:1875] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.035101 3488 ts_tablet_manager.cc:1888] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.035176 3488 log.cc:1196] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/ea8c5159a1e24471ab878ffb50a6df05
I20240502 14:00:23.035435 3488 ts_tablet_manager.cc:1909] T ea8c5159a1e24471ab878ffb50a6df05 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:23.035599 522 tablet_service.cc:1514] Processing DeleteTablet for tablet 2432b3ab36a54fbfa108031f67634e2c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:23.035677 3485 ts_tablet_manager.cc:1888] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.035744 3485 log.cc:1196] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/2432b3ab36a54fbfa108031f67634e2c
I20240502 14:00:23.036038 3485 ts_tablet_manager.cc:1909] T 2432b3ab36a54fbfa108031f67634e2c P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:23.036731 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 2432b3ab36a54fbfa108031f67634e2c (table rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]) successfully deleted
I20240502 14:00:23.036904 386 tablet_service.cc:1514] Processing DeleteTablet for tablet aa30d86ef587427db4213f5c178a744a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:23.036996 387 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8c5159a1e24471ab878ffb50a6df05 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:23.036976 653 tablet_service.cc:1514] Processing DeleteTablet for tablet aa30d86ef587427db4213f5c178a744a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:23.037075 3485 tablet_replica.cc:330] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:23.037166 3484 tablet_replica.cc:330] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:23.037220 3486 ts_tablet_manager.cc:1888] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.037227 3485 raft_consensus.cc:2229] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:23.037282 3486 log.cc:1196] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/ea8c5159a1e24471ab878ffb50a6df05
I20240502 14:00:23.037298 3484 raft_consensus.cc:2229] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.037400 3485 raft_consensus.cc:2258] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.037423 3484 raft_consensus.cc:2258] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.037528 3486 ts_tablet_manager.cc:1909] T ea8c5159a1e24471ab878ffb50a6df05 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:23.037906 3484 ts_tablet_manager.cc:1875] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.038076 3485 ts_tablet_manager.cc:1875] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.038167 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet ea8c5159a1e24471ab878ffb50a6df05 (table rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]) successfully deleted
I20240502 14:00:23.038432 3486 tablet_replica.cc:330] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:23.038548 3486 raft_consensus.cc:2229] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:23.038693 3486 raft_consensus.cc:2258] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.039137 3486 ts_tablet_manager.cc:1875] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.040105 3485 ts_tablet_manager.cc:1888] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.040555 3484 ts_tablet_manager.cc:1888] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.040633 3484 log.cc:1196] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/aa30d86ef587427db4213f5c178a744a
I20240502 14:00:23.040800 3485 log.cc:1196] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/aa30d86ef587427db4213f5c178a744a
I20240502 14:00:23.040941 3484 ts_tablet_manager.cc:1909] T aa30d86ef587427db4213f5c178a744a P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:23.041133 3485 ts_tablet_manager.cc:1909] T aa30d86ef587427db4213f5c178a744a P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:23.041432 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 2432b3ab36a54fbfa108031f67634e2c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:23.041582 784 tablet_service.cc:1514] Processing DeleteTablet for tablet aa30d86ef587427db4213f5c178a744a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:23.041749 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet aa30d86ef587427db4213f5c178a744a (table rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]) successfully deleted
I20240502 14:00:23.041913 3485 tablet_replica.cc:330] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:23.042044 3485 raft_consensus.cc:2229] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.042068 3486 ts_tablet_manager.cc:1888] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.042125 3486 log.cc:1196] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/2432b3ab36a54fbfa108031f67634e2c
I20240502 14:00:23.042174 3485 raft_consensus.cc:2258] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.042385 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet aa30d86ef587427db4213f5c178a744a (table rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]) successfully deleted
I20240502 14:00:23.042405 3488 tablet_replica.cc:330] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:23.042495 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet ea8c5159a1e24471ab878ffb50a6df05 (table rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]) successfully deleted
I20240502 14:00:23.042647 3485 ts_tablet_manager.cc:1875] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.042678 3488 raft_consensus.cc:2229] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.042779 3486 ts_tablet_manager.cc:1909] T 2432b3ab36a54fbfa108031f67634e2c P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:23.042883 3488 raft_consensus.cc:2258] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.043367 3488 ts_tablet_manager.cc:1875] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.043486 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 2432b3ab36a54fbfa108031f67634e2c (table rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]) successfully deleted
I20240502 14:00:23.044543 3485 ts_tablet_manager.cc:1888] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.044602 3485 log.cc:1196] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/ea8c5159a1e24471ab878ffb50a6df05
I20240502 14:00:23.044816 3485 ts_tablet_manager.cc:1909] T ea8c5159a1e24471ab878ffb50a6df05 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:23.045387 3488 ts_tablet_manager.cc:1888] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.045506 3488 log.cc:1196] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/2432b3ab36a54fbfa108031f67634e2c
I20240502 14:00:23.045648 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet ea8c5159a1e24471ab878ffb50a6df05 (table rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]) successfully deleted
I20240502 14:00:23.045799 3488 ts_tablet_manager.cc:1909] T 2432b3ab36a54fbfa108031f67634e2c P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:23.046427 3488 tablet_replica.cc:330] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:23.046509 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 2432b3ab36a54fbfa108031f67634e2c (table rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]) successfully deleted
I20240502 14:00:23.046586 3488 raft_consensus.cc:2229] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.046696 3488 raft_consensus.cc:2258] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.047082 3488 ts_tablet_manager.cc:1875] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.048609 3488 ts_tablet_manager.cc:1888] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.048671 3488 log.cc:1196] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/aa30d86ef587427db4213f5c178a744a
I20240502 14:00:23.048895 3488 ts_tablet_manager.cc:1909] T aa30d86ef587427db4213f5c178a744a P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:23.049407 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet aa30d86ef587427db4213f5c178a744a (table rebalance_test_table_2 [id=0701f0c3be0e4a00bee8ce1ed8534336]) successfully deleted
I20240502 14:00:23.072921 3352 raft_consensus.cc:482] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:23.073073 3352 raft_consensus.cc:504] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:23.074069 3503 consensus_queue.cc:1035] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.074182 3352 leader_election.cc:288] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:23.077378 3503 consensus_queue.cc:1035] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.082123 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54618de152b04381ae6c792a57d0a9e6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:23.082325 2884 raft_consensus.cc:2454] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:23.083083 3087 leader_election.cc:302] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:23.083323 3352 raft_consensus.cc:2781] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:23.083375 3352 raft_consensus.cc:482] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:23.083418 3352 raft_consensus.cc:3037] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.084529 3352 raft_consensus.cc:504] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:23.084918 3352 leader_election.cc:288] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:23.085274 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54618de152b04381ae6c792a57d0a9e6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:23.085393 2884 raft_consensus.cc:3037] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.086398 2884 raft_consensus.cc:2454] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:23.086356 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54618de152b04381ae6c792a57d0a9e6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:23.086360 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54618de152b04381ae6c792a57d0a9e6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:23.086513 3284 raft_consensus.cc:2454] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:23.086714 3087 leader_election.cc:302] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:23.087016 3352 raft_consensus.cc:2781] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:23.087172 3352 raft_consensus.cc:686] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:23.087383 3352 consensus_queue.cc:237] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:23.089850 32732 catalog_manager.cc:5484] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:23.191236 3504 consensus_queue.cc:1035] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.194816 3504 consensus_queue.cc:1035] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.247757 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:23.254243 3284 raft_consensus.cc:3037] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.255656 3284 raft_consensus.cc:1261] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:23.256475 3352 consensus_queue.cc:1035] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.259132 2884 raft_consensus.cc:1261] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:23.259848 3352 consensus_queue.cc:1035] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.261226 32726 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 54618de152b04381ae6c792a57d0a9e6 (table rebalancer_dynamic_table [id=8f94f7b869a340139441aa113c7682a1]). Expected version 1 got 0
I20240502 14:00:23.261901 3577 tablet.cc:1722] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:23.262334 3577 tablet.cc:1620] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:23.263316 3578 tablet.cc:1722] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:23.263906 32726 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 54618de152b04381ae6c792a57d0a9e6 (table rebalancer_dynamic_table [id=8f94f7b869a340139441aa113c7682a1]). Expected version 1 got 0
I20240502 14:00:23.264183 3578 tablet.cc:1620] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:23.264529 3568 mvcc.cc:205] Tried to move back new op lower bound from 7023240901642190848 to 7023240900968509440. Current Snapshot: MvccSnapshot[applied={T|T < 7023240901642190848}]
I20240502 14:00:23.268554 3566 mvcc.cc:205] Tried to move back new op lower bound from 7023240901642190848 to 7023240900968509440. Current Snapshot: MvccSnapshot[applied={T|T < 7023240901642190848}]
I20240502 14:00:23.268745 32731 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet 54618de152b04381ae6c792a57d0a9e6 (table rebalancer_dynamic_table [id=8f94f7b869a340139441aa113c7682a1]). Expected version 1 got 0
I20240502 14:00:23.269263 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=8f94f7b869a340139441aa113c7682a1] alter complete (version 1)
I20240502 14:00:23.269892 3583 tablet.cc:1722] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:23.270597 3583 tablet.cc:1620] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:23.323990 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:23.330260 2865 tablet_service.cc:1467] Processing CreateTablet for tablet 923a5d2ccd0448b381d88ced69ae8fe2 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:23.330647 2865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923a5d2ccd0448b381d88ced69ae8fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.330747 387 tablet_service.cc:1467] Processing CreateTablet for tablet a176c4678de94ccc93b14d04fc46ea90 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:23.331040 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a176c4678de94ccc93b14d04fc46ea90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.331974 3130 tablet_service.cc:1467] Processing CreateTablet for tablet a176c4678de94ccc93b14d04fc46ea90 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:23.332285 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a176c4678de94ccc93b14d04fc46ea90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.333074 3584 tablet_bootstrap.cc:492] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:23.330260 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 10c70369a07e45afb7bd885b58daec40 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:23.333245 784 tablet_service.cc:1467] Processing CreateTablet for tablet 10c70369a07e45afb7bd885b58daec40 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:23.333474 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10c70369a07e45afb7bd885b58daec40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.333500 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10c70369a07e45afb7bd885b58daec40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.333928 3584 tablet_bootstrap.cc:654] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.335458 3495 tablet_bootstrap.cc:492] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:23.335783 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 10c70369a07e45afb7bd885b58daec40 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:23.336083 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10c70369a07e45afb7bd885b58daec40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.336117 3584 tablet_bootstrap.cc:492] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:23.336086 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 923a5d2ccd0448b381d88ced69ae8fe2 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:23.336194 3584 ts_tablet_manager.cc:1367] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:23.336320 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923a5d2ccd0448b381d88ced69ae8fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.336369 3495 tablet_bootstrap.cc:654] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.336606 3584 raft_consensus.cc:348] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:23.336702 3584 raft_consensus.cc:374] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.336730 3584 raft_consensus.cc:729] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.336827 3584 consensus_queue.cc:260] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:23.337129 3584 ts_tablet_manager.cc:1398] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:23.337559 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 923a5d2ccd0448b381d88ced69ae8fe2 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:23.337778 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923a5d2ccd0448b381d88ced69ae8fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.338243 3349 tablet_bootstrap.cc:492] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:23.338303 3588 tablet_bootstrap.cc:492] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:23.339150 3349 tablet_bootstrap.cc:654] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.339167 3588 tablet_bootstrap.cc:654] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.340373 3349 tablet_bootstrap.cc:492] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:23.340454 3349 ts_tablet_manager.cc:1367] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:23.335786 785 tablet_service.cc:1467] Processing CreateTablet for tablet a176c4678de94ccc93b14d04fc46ea90 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:23.340917 3349 raft_consensus.cc:348] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:23.340999 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a176c4678de94ccc93b14d04fc46ea90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.341024 3349 raft_consensus.cc:374] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.341055 3349 raft_consensus.cc:729] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.341187 3349 consensus_queue.cc:260] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:23.341430 3349 ts_tablet_manager.cc:1398] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:23.341581 3349 tablet_bootstrap.cc:492] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:23.341887 3586 tablet_bootstrap.cc:492] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:23.342820 3586 tablet_bootstrap.cc:654] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.344841 3349 tablet_bootstrap.cc:654] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.345638 3495 tablet_bootstrap.cc:492] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:23.345736 3495 ts_tablet_manager.cc:1367] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:23.345775 3586 tablet_bootstrap.cc:492] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:23.345854 3586 ts_tablet_manager.cc:1367] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:23.346153 3349 tablet_bootstrap.cc:492] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:23.346220 3349 ts_tablet_manager.cc:1367] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:23.346235 3586 raft_consensus.cc:348] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:23.346210 3495 raft_consensus.cc:348] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:23.346334 3586 raft_consensus.cc:374] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.346335 3495 raft_consensus.cc:374] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.346367 3495 raft_consensus.cc:729] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.346367 3586 raft_consensus.cc:729] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.346491 3495 consensus_queue.cc:260] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:23.346493 3586 consensus_queue.cc:260] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:23.346622 3349 raft_consensus.cc:348] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:23.346735 3349 raft_consensus.cc:374] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.346755 3495 ts_tablet_manager.cc:1398] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:23.346765 3349 raft_consensus.cc:729] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.346853 3586 ts_tablet_manager.cc:1398] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:23.346920 3495 tablet_bootstrap.cc:492] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:23.346884 3349 consensus_queue.cc:260] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:23.346987 3586 tablet_bootstrap.cc:492] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:23.347350 3349 ts_tablet_manager.cc:1398] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:23.347756 3495 tablet_bootstrap.cc:654] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.347756 3586 tablet_bootstrap.cc:654] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.349668 3504 consensus_queue.cc:1035] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.352110 3588 tablet_bootstrap.cc:492] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:23.352219 3588 ts_tablet_manager.cc:1367] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 14:00:23.354276 3504 consensus_queue.cc:1035] T 9004b8cfea064786a1522e7ced149167 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.356003 3588 raft_consensus.cc:348] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:23.356151 3588 raft_consensus.cc:374] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.356192 3588 raft_consensus.cc:729] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.356335 3588 consensus_queue.cc:260] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:23.356739 3588 ts_tablet_manager.cc:1398] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:23.357002 3588 tablet_bootstrap.cc:492] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:23.358206 3588 tablet_bootstrap.cc:654] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.359544 3588 tablet_bootstrap.cc:492] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:23.359550 3495 tablet_bootstrap.cc:492] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:23.359550 3586 tablet_bootstrap.cc:492] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:23.359630 3588 ts_tablet_manager.cc:1367] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:23.359637 3586 ts_tablet_manager.cc:1367] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 14:00:23.359637 3495 ts_tablet_manager.cc:1367] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 14:00:23.360080 3588 raft_consensus.cc:348] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:23.360100 3495 raft_consensus.cc:348] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:23.360100 3586 raft_consensus.cc:348] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:23.360208 3495 raft_consensus.cc:374] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.360213 3588 raft_consensus.cc:374] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.360226 3586 raft_consensus.cc:374] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.360245 3495 raft_consensus.cc:729] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.360263 3586 raft_consensus.cc:729] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.360278 3588 raft_consensus.cc:729] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.360383 3495 consensus_queue.cc:260] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:23.360383 3586 consensus_queue.cc:260] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:23.360417 3588 consensus_queue.cc:260] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:23.360642 3495 ts_tablet_manager.cc:1398] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:23.360653 3586 ts_tablet_manager.cc:1398] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:23.362352 3588 ts_tablet_manager.cc:1398] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:23.370572 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:23.376605 3577 tablet.cc:1722] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:23.376956 3577 tablet.cc:1620] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:23.376896 3583 tablet.cc:1722] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:23.377012 3578 tablet.cc:1722] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:23.377205 3583 tablet.cc:1620] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:23.377313 3578 tablet.cc:1620] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:23.385205 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=8f94f7b869a340139441aa113c7682a1] alter complete (version 2)
I20240502 14:00:23.451941 3423 raft_consensus.cc:482] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:23.452075 3423 raft_consensus.cc:504] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:23.452533 3423 leader_election.cc:288] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:23.453364 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923a5d2ccd0448b381d88ced69ae8fe2" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:23.453552 3152 raft_consensus.cc:2454] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:23.453912 2954 leader_election.cc:302] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:23.454088 3423 raft_consensus.cc:2781] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:23.454133 3423 raft_consensus.cc:482] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:23.454185 3423 raft_consensus.cc:3037] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.455318 3423 raft_consensus.cc:504] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:23.455664 3423 leader_election.cc:288] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:23.456070 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923a5d2ccd0448b381d88ced69ae8fe2" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:23.456192 3152 raft_consensus.cc:3037] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.457017 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923a5d2ccd0448b381d88ced69ae8fe2" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:23.457139 2884 raft_consensus.cc:2454] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:23.457096 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923a5d2ccd0448b381d88ced69ae8fe2" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:23.457221 2886 raft_consensus.cc:3037] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.457278 3152 raft_consensus.cc:2454] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:23.457568 2954 leader_election.cc:302] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:23.457783 3423 raft_consensus.cc:2781] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:23.457859 3423 raft_consensus.cc:686] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:23.458001 3423 consensus_queue.cc:237] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:23.458122 2886 raft_consensus.cc:2454] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:23.459754 32732 catalog_manager.cc:5484] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:23.479730 3423 raft_consensus.cc:482] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:23.479885 3423 raft_consensus.cc:504] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:23.480366 3423 leader_election.cc:288] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:23.481175 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c70369a07e45afb7bd885b58daec40" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:23.481328 2886 raft_consensus.cc:2454] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:23.481657 2954 leader_election.cc:302] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:23.481848 3423 raft_consensus.cc:2781] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:23.481915 3423 raft_consensus.cc:482] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:23.481969 3423 raft_consensus.cc:3037] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.482755 3423 raft_consensus.cc:504] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:23.483382 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c70369a07e45afb7bd885b58daec40" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:23.483443 3423 leader_election.cc:288] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:23.483507 2886 raft_consensus.cc:3037] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.484477 2886 raft_consensus.cc:2454] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:23.484720 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c70369a07e45afb7bd885b58daec40" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:23.484808 2954 leader_election.cc:302] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:23.484874 804 raft_consensus.cc:2454] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:23.484730 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c70369a07e45afb7bd885b58daec40" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:23.485062 806 raft_consensus.cc:3037] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.485765 806 raft_consensus.cc:2454] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:23.486064 3423 raft_consensus.cc:2781] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:23.486181 3423 raft_consensus.cc:686] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:23.486325 3423 consensus_queue.cc:237] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:23.487923 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:23.488077 32732 catalog_manager.cc:5484] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:23.494009 3577 tablet.cc:1722] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:23.494374 3578 tablet.cc:1722] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:23.494400 3583 tablet.cc:1722] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:23.500319 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=8f94f7b869a340139441aa113c7682a1] alter complete (version 3)
I20240502 14:00:23.513716 2550 raft_consensus.cc:482] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:23.513845 2550 raft_consensus.cc:504] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:23.514509 2550 leader_election.cc:288] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:23.514811 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a176c4678de94ccc93b14d04fc46ea90" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:23.514971 804 raft_consensus.cc:2454] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:23.515265 344 leader_election.cc:302] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:23.515422 2550 raft_consensus.cc:2781] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:23.515497 2550 raft_consensus.cc:482] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:23.515547 2550 raft_consensus.cc:3037] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.516419 2550 raft_consensus.cc:504] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:23.516703 2550 leader_election.cc:288] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:23.517043 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a176c4678de94ccc93b14d04fc46ea90" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:23.517158 804 raft_consensus.cc:3037] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.517901 804 raft_consensus.cc:2454] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:23.518208 344 leader_election.cc:302] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:23.518365 2550 raft_consensus.cc:2781] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:23.518465 2550 raft_consensus.cc:686] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:23.518620 2550 consensus_queue.cc:237] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:23.520138 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a176c4678de94ccc93b14d04fc46ea90" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:23.520179 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a176c4678de94ccc93b14d04fc46ea90" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:23.520287 3152 raft_consensus.cc:2454] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:23.520938 32731 catalog_manager.cc:5484] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:23.542274 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_4" } modify_external_catalogs: true
I20240502 14:00:23.542483 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_4" } modify_external_catalogs: true
I20240502 14:00:23.544749 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 2bef2d8b7cbf45548d64bf94560bed3f
I20240502 14:00:23.545131 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 00e31a2c43dd4f6b841a2c00fe61875e
I20240502 14:00:23.545418 387 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bef2d8b7cbf45548d64bf94560bed3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:23.545452 522 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bef2d8b7cbf45548d64bf94560bed3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:23.545565 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 89dc41d04bab4cc09f6c17a5d8524358
I20240502 14:00:23.545637 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bef2d8b7cbf45548d64bf94560bed3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:23.545703 3485 tablet_replica.cc:330] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:23.545854 3485 raft_consensus.cc:2229] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.545958 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 00e31a2c43dd4f6b841a2c00fe61875e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:23.546030 3485 raft_consensus.cc:2258] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.546173 3488 tablet_replica.cc:330] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:23.546321 3488 raft_consensus.cc:2229] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.546329 386 tablet_service.cc:1514] Processing DeleteTablet for tablet 89dc41d04bab4cc09f6c17a5d8524358 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:23.546456 3488 raft_consensus.cc:2258] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.546474 387 tablet_service.cc:1514] Processing DeleteTablet for tablet 00e31a2c43dd4f6b841a2c00fe61875e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:23.546715 3485 ts_tablet_manager.cc:1875] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.546912 3608 tablet_replica.cc:330] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:23.546969 3488 ts_tablet_manager.cc:1875] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.547103 3608 raft_consensus.cc:2229] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:23.547284 3608 raft_consensus.cc:2258] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.547864 3608 ts_tablet_manager.cc:1875] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.549104 3485 ts_tablet_manager.cc:1888] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.549168 3485 log.cc:1196] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/2bef2d8b7cbf45548d64bf94560bed3f
I20240502 14:00:23.549180 3488 ts_tablet_manager.cc:1888] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.549240 3488 log.cc:1196] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/00e31a2c43dd4f6b841a2c00fe61875e
I20240502 14:00:23.549441 3485 ts_tablet_manager.cc:1909] T 2bef2d8b7cbf45548d64bf94560bed3f P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:23.549515 3488 ts_tablet_manager.cc:1909] T 00e31a2c43dd4f6b841a2c00fe61875e P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:23.550026 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 89dc41d04bab4cc09f6c17a5d8524358 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:23.550398 3488 tablet_replica.cc:330] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:23.550341 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 00e31a2c43dd4f6b841a2c00fe61875e (table rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]) successfully deleted
I20240502 14:00:23.550560 3488 raft_consensus.cc:2229] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:23.550727 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 2bef2d8b7cbf45548d64bf94560bed3f (table rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]) successfully deleted
I20240502 14:00:23.550817 3488 raft_consensus.cc:2258] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.550832 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 89dc41d04bab4cc09f6c17a5d8524358 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:23.550976 3485 tablet_replica.cc:330] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:23.551007 522 tablet_service.cc:1514] Processing DeleteTablet for tablet 00e31a2c43dd4f6b841a2c00fe61875e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:23.551069 3609 tablet_replica.cc:330] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:23.551136 3485 raft_consensus.cc:2229] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.551206 3609 raft_consensus.cc:2229] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.551265 3485 raft_consensus.cc:2258] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.551380 3609 raft_consensus.cc:2258] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.551687 3488 ts_tablet_manager.cc:1875] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.551730 3485 ts_tablet_manager.cc:1875] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.552009 3609 ts_tablet_manager.cc:1875] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.552013 3608 ts_tablet_manager.cc:1888] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.552083 3608 log.cc:1196] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/2bef2d8b7cbf45548d64bf94560bed3f
I20240502 14:00:23.552420 3608 ts_tablet_manager.cc:1909] T 2bef2d8b7cbf45548d64bf94560bed3f P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:23.553022 3608 tablet_replica.cc:330] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:23.553164 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 2bef2d8b7cbf45548d64bf94560bed3f (table rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]) successfully deleted
I20240502 14:00:23.553208 3608 raft_consensus.cc:2229] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:23.553445 3608 raft_consensus.cc:2258] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.553757 3488 ts_tablet_manager.cc:1888] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.553874 3488 log.cc:1196] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/89dc41d04bab4cc09f6c17a5d8524358
I20240502 14:00:23.554028 3608 ts_tablet_manager.cc:1875] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.554090 3485 ts_tablet_manager.cc:1888] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.554240 3488 ts_tablet_manager.cc:1909] T 89dc41d04bab4cc09f6c17a5d8524358 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:23.554248 3485 log.cc:1196] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/89dc41d04bab4cc09f6c17a5d8524358
I20240502 14:00:23.554574 3485 ts_tablet_manager.cc:1909] T 89dc41d04bab4cc09f6c17a5d8524358 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:23.554672 3609 ts_tablet_manager.cc:1888] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.554805 3609 log.cc:1196] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/2bef2d8b7cbf45548d64bf94560bed3f
I20240502 14:00:23.555124 3609 ts_tablet_manager.cc:1909] T 2bef2d8b7cbf45548d64bf94560bed3f P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:23.555131 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 89dc41d04bab4cc09f6c17a5d8524358 (table rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]) successfully deleted
I20240502 14:00:23.555460 3485 tablet_replica.cc:330] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:23.555608 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 89dc41d04bab4cc09f6c17a5d8524358 (table rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]) successfully deleted
I20240502 14:00:23.555672 3485 raft_consensus.cc:2229] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.555799 3485 raft_consensus.cc:2258] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.555855 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 2bef2d8b7cbf45548d64bf94560bed3f (table rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]) successfully deleted
I20240502 14:00:23.556092 3609 tablet_replica.cc:330] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:23.556255 3609 raft_consensus.cc:2229] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.556389 3609 raft_consensus.cc:2258] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.556439 3485 ts_tablet_manager.cc:1875] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.556620 3608 ts_tablet_manager.cc:1888] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.556767 3608 log.cc:1196] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/00e31a2c43dd4f6b841a2c00fe61875e
I20240502 14:00:23.557039 3608 ts_tablet_manager.cc:1909] T 00e31a2c43dd4f6b841a2c00fe61875e P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:23.557039 3609 ts_tablet_manager.cc:1875] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.557703 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 00e31a2c43dd4f6b841a2c00fe61875e (table rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]) successfully deleted
I20240502 14:00:23.558182 3485 ts_tablet_manager.cc:1888] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.558249 3485 log.cc:1196] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/00e31a2c43dd4f6b841a2c00fe61875e
I20240502 14:00:23.558467 3485 ts_tablet_manager.cc:1909] T 00e31a2c43dd4f6b841a2c00fe61875e P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:23.558614 3609 ts_tablet_manager.cc:1888] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:23.558677 3609 log.cc:1196] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/89dc41d04bab4cc09f6c17a5d8524358
I20240502 14:00:23.558861 3609 ts_tablet_manager.cc:1909] T 89dc41d04bab4cc09f6c17a5d8524358 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:23.559113 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 00e31a2c43dd4f6b841a2c00fe61875e (table rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]) successfully deleted
I20240502 14:00:23.559396 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 89dc41d04bab4cc09f6c17a5d8524358 (table rebalance_test_table_4 [id=85a2fa13ee62482eaa13e1fd02251f4b]) successfully deleted
I20240502 14:00:23.603583 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:23.603778 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:23.606209 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 54618de152b04381ae6c792a57d0a9e6
I20240502 14:00:23.606966 3262 tablet_service.cc:1514] Processing DeleteTablet for tablet 54618de152b04381ae6c792a57d0a9e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:23.606950 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 54618de152b04381ae6c792a57d0a9e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:23.606995 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 54618de152b04381ae6c792a57d0a9e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:23 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:23.607367 3610 tablet_replica.cc:330] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:23.607472 3612 tablet_replica.cc:330] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:23.607477 3611 tablet_replica.cc:330] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:23.607573 3610 raft_consensus.cc:2229] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:23.607775 3612 raft_consensus.cc:2229] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.607776 3611 raft_consensus.cc:2229] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:23.607896 3610 raft_consensus.cc:2258] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.608011 3612 raft_consensus.cc:2258] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.608115 3611 raft_consensus.cc:2258] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:23.609010 3610 ts_tablet_manager.cc:1875] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.609174 3612 ts_tablet_manager.cc:1875] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.609225 3611 ts_tablet_manager.cc:1875] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:23.611606 3611 ts_tablet_manager.cc:1888] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:23.611764 3611 log.cc:1196] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/54618de152b04381ae6c792a57d0a9e6
I20240502 14:00:23.611871 3612 ts_tablet_manager.cc:1888] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:23.612015 3612 log.cc:1196] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/54618de152b04381ae6c792a57d0a9e6
I20240502 14:00:23.612015 3610 ts_tablet_manager.cc:1888] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:23.612087 3610 log.cc:1196] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/54618de152b04381ae6c792a57d0a9e6
I20240502 14:00:23.612340 3611 ts_tablet_manager.cc:1909] T 54618de152b04381ae6c792a57d0a9e6 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:23.612402 3610 ts_tablet_manager.cc:1909] T 54618de152b04381ae6c792a57d0a9e6 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:23.612778 3612 ts_tablet_manager.cc:1909] T 54618de152b04381ae6c792a57d0a9e6 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting consensus metadata
I20240502 14:00:23.613193 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 54618de152b04381ae6c792a57d0a9e6 (table rebalancer_dynamic_table_new_name [id=8f94f7b869a340139441aa113c7682a1]) successfully deleted
I20240502 14:00:23.613476 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 54618de152b04381ae6c792a57d0a9e6 (table rebalancer_dynamic_table_new_name [id=8f94f7b869a340139441aa113c7682a1]) successfully deleted
I20240502 14:00:23.613821 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 54618de152b04381ae6c792a57d0a9e6 (table rebalancer_dynamic_table_new_name [id=8f94f7b869a340139441aa113c7682a1]) successfully deleted
I20240502 14:00:23.708046 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:23.713438 522 tablet_service.cc:1467] Processing CreateTablet for tablet b379f016bee14bb29b6dd7033236a28a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=52e059b3549241d59ab5da3a91ab86d2]), partition=
I20240502 14:00:23.713444 3262 tablet_service.cc:1467] Processing CreateTablet for tablet b379f016bee14bb29b6dd7033236a28a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=52e059b3549241d59ab5da3a91ab86d2]), partition=
I20240502 14:00:23.713721 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b379f016bee14bb29b6dd7033236a28a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.713721 3262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b379f016bee14bb29b6dd7033236a28a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.713863 2866 tablet_service.cc:1467] Processing CreateTablet for tablet b379f016bee14bb29b6dd7033236a28a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=52e059b3549241d59ab5da3a91ab86d2]), partition=
I20240502 14:00:23.714103 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b379f016bee14bb29b6dd7033236a28a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:23.715806 3495 tablet_bootstrap.cc:492] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:23.716125 3615 tablet_bootstrap.cc:492] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:23.716125 3614 tablet_bootstrap.cc:492] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:23.716858 3495 tablet_bootstrap.cc:654] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.716953 3615 tablet_bootstrap.cc:654] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.716953 3614 tablet_bootstrap.cc:654] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:23.717936 3495 tablet_bootstrap.cc:492] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:23.718029 3495 ts_tablet_manager.cc:1367] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 14:00:23.718055 3615 tablet_bootstrap.cc:492] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:23.718171 3615 ts_tablet_manager.cc:1367] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:23.718508 3495 raft_consensus.cc:348] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:23.718627 3495 raft_consensus.cc:374] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.718585 3615 raft_consensus.cc:348] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:23.718658 3495 raft_consensus.cc:729] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.718713 3615 raft_consensus.cc:374] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.718775 3615 raft_consensus.cc:729] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.718770 3495 consensus_queue.cc:260] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:23.718912 3615 consensus_queue.cc:260] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:23.719189 3495 ts_tablet_manager.cc:1398] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:23.719224 3615 ts_tablet_manager.cc:1398] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:23.720137 3614 tablet_bootstrap.cc:492] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:23.720243 3614 ts_tablet_manager.cc:1367] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:23.720917 3614 raft_consensus.cc:348] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:23.721026 3614 raft_consensus.cc:374] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:23.721060 3614 raft_consensus.cc:729] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:23.721350 3614 consensus_queue.cc:260] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:23.721664 3614 ts_tablet_manager.cc:1398] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:23.747519 3613 raft_consensus.cc:482] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:23.747642 3613 raft_consensus.cc:504] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:23.748225 3613 leader_election.cc:288] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:23.752776 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b379f016bee14bb29b6dd7033236a28a" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:23.752945 2886 raft_consensus.cc:2454] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:23.752976 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b379f016bee14bb29b6dd7033236a28a" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:23.753113 541 raft_consensus.cc:2454] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:23.753238 3219 leader_election.cc:302] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:23.753512 3613 raft_consensus.cc:2781] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:23.753594 3613 raft_consensus.cc:482] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:23.753635 3613 raft_consensus.cc:3037] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.754519 3613 raft_consensus.cc:504] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:23.754905 3613 leader_election.cc:288] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:23.755257 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b379f016bee14bb29b6dd7033236a28a" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:23.755261 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b379f016bee14bb29b6dd7033236a28a" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:23.755380 2886 raft_consensus.cc:3037] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.755380 541 raft_consensus.cc:3037] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:23.756410 2886 raft_consensus.cc:2454] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:23.756410 541 raft_consensus.cc:2454] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:23.756726 3216 leader_election.cc:302] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:23.756896 3613 raft_consensus.cc:2781] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:23.757016 3613 raft_consensus.cc:686] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:23.757201 3613 consensus_queue.cc:237] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:23.759061 32730 catalog_manager.cc:5484] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:23.871131 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:23.875361 541 raft_consensus.cc:1261] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:23.875525 2886 raft_consensus.cc:1261] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:23.876055 3556 consensus_queue.cc:1035] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.876399 3613 consensus_queue.cc:1035] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.878561 32730 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet b379f016bee14bb29b6dd7033236a28a (table rebalancer_dynamic_table [id=52e059b3549241d59ab5da3a91ab86d2]). Expected version 1 got 0
I20240502 14:00:23.879055 32732 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet b379f016bee14bb29b6dd7033236a28a (table rebalancer_dynamic_table [id=52e059b3549241d59ab5da3a91ab86d2]). Expected version 1 got 0
I20240502 14:00:23.879914 3578 tablet.cc:1722] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:23.880259 3578 tablet.cc:1620] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:23.880667 3583 tablet.cc:1722] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:23.880923 3583 tablet.cc:1620] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:23.881944 32731 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet b379f016bee14bb29b6dd7033236a28a (table rebalancer_dynamic_table [id=52e059b3549241d59ab5da3a91ab86d2]). Expected version 1 got 0
I20240502 14:00:23.882804 3627 tablet.cc:1722] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:23.883284 3627 tablet.cc:1620] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:23.883332 32732 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet b379f016bee14bb29b6dd7033236a28a (table rebalancer_dynamic_table [id=52e059b3549241d59ab5da3a91ab86d2]). Expected version 1 got 0
I20240502 14:00:23.884549 3423 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.886740 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=52e059b3549241d59ab5da3a91ab86d2] alter complete (version 1)
I20240502 14:00:23.888669 3423 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.890388 3423 consensus_queue.cc:1035] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.893376 3423 consensus_queue.cc:1035] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:23.989076 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:23.994618 3578 tablet.cc:1722] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:23.995025 3578 tablet.cc:1620] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:23.995157 3583 tablet.cc:1722] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:23.995447 3583 tablet.cc:1620] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:23.996851 3627 tablet.cc:1722] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:23.997200 3627 tablet.cc:1620] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:24.000499 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=52e059b3549241d59ab5da3a91ab86d2] alter complete (version 2)
I20240502 14:00:24.006678 2550 consensus_queue.cc:1035] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:24.009485 3151 raft_consensus.cc:3037] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.011092 2550 consensus_queue.cc:1035] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:24.038159 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:24.044663 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 90c02b136d2c492bb727e77793a1db20 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=13b33238c8214cd6b88686b2b94abdb2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:24.044839 522 tablet_service.cc:1467] Processing CreateTablet for tablet 90c02b136d2c492bb727e77793a1db20 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=13b33238c8214cd6b88686b2b94abdb2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:24.044839 3262 tablet_service.cc:1467] Processing CreateTablet for tablet 90c02b136d2c492bb727e77793a1db20 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=13b33238c8214cd6b88686b2b94abdb2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:24.045024 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c02b136d2c492bb727e77793a1db20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.045120 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c02b136d2c492bb727e77793a1db20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.045120 3262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c02b136d2c492bb727e77793a1db20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.047227 3615 tablet_bootstrap.cc:492] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:24.047302 3614 tablet_bootstrap.cc:492] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:24.048162 3615 tablet_bootstrap.cc:654] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.048206 3614 tablet_bootstrap.cc:654] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.048307 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 2d9d248c2c6745489d2656752db9ddd5 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=13b33238c8214cd6b88686b2b94abdb2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:24.048586 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d9d248c2c6745489d2656752db9ddd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.049175 3615 tablet_bootstrap.cc:492] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:24.049268 3615 ts_tablet_manager.cc:1367] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:24.049477 3132 tablet_service.cc:1467] Processing CreateTablet for tablet c2ce30696a474d23818c3f70cc86e9dd (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=13b33238c8214cd6b88686b2b94abdb2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:24.049700 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ce30696a474d23818c3f70cc86e9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.049639 32730 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_6" } modify_external_catalogs: true
I20240502 14:00:24.049757 32730 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_6" } modify_external_catalogs: true
I20240502 14:00:24.049739 3615 raft_consensus.cc:348] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.049854 3615 raft_consensus.cc:374] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.049887 3615 raft_consensus.cc:729] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.050012 3615 consensus_queue.cc:260] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.050276 3615 ts_tablet_manager.cc:1398] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:24.051054 2999 tablet_service.cc:1467] Processing CreateTablet for tablet c2ce30696a474d23818c3f70cc86e9dd (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=13b33238c8214cd6b88686b2b94abdb2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:24.051285 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ce30696a474d23818c3f70cc86e9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.051651 3614 tablet_bootstrap.cc:492] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:24.051745 3614 ts_tablet_manager.cc:1367] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:24.052167 3614 raft_consensus.cc:348] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.052250 3614 raft_consensus.cc:374] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.052271 3614 raft_consensus.cc:729] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.052377 3614 consensus_queue.cc:260] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.052630 3614 ts_tablet_manager.cc:1398] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:24.053695 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 2d9d248c2c6745489d2656752db9ddd5 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=13b33238c8214cd6b88686b2b94abdb2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:24.053968 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d9d248c2c6745489d2656752db9ddd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.054031 3637 tablet_bootstrap.cc:492] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:24.055063 3637 tablet_bootstrap.cc:654] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.056162 3495 tablet_bootstrap.cc:492] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:24.057145 3495 tablet_bootstrap.cc:654] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.058007 3262 tablet_service.cc:1467] Processing CreateTablet for tablet 2d9d248c2c6745489d2656752db9ddd5 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=13b33238c8214cd6b88686b2b94abdb2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:24.058208 32730 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet a3e7aa78b21a42f89cd3ac29cd113206
I20240502 14:00:24.058248 3262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d9d248c2c6745489d2656752db9ddd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.058493 32730 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 8122e08018cb40e987cdd4e9053c6cbd
I20240502 14:00:24.058629 32730 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 6b96ddb6ec9f4ef0bda0d354fc39adec
I20240502 14:00:24.058884 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 8122e08018cb40e987cdd4e9053c6cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:24.059091 3609 tablet_replica.cc:330] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:24.059114 522 tablet_service.cc:1467] Processing CreateTablet for tablet c2ce30696a474d23818c3f70cc86e9dd (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=13b33238c8214cd6b88686b2b94abdb2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:24.059230 3609 raft_consensus.cc:2229] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:24.059370 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ce30696a474d23818c3f70cc86e9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.059547 3609 raft_consensus.cc:2258] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.060087 3609 ts_tablet_manager.cc:1875] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.060175 521 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b96ddb6ec9f4ef0bda0d354fc39adec with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:24.061252 3640 tablet_bootstrap.cc:492] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:24.061471 3615 tablet_bootstrap.cc:492] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:24.061996 785 tablet_service.cc:1514] Processing DeleteTablet for tablet a3e7aa78b21a42f89cd3ac29cd113206 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:24.062144 520 tablet_service.cc:1514] Processing DeleteTablet for tablet a3e7aa78b21a42f89cd3ac29cd113206 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:24.062239 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 8122e08018cb40e987cdd4e9053c6cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:24.062348 3614 tablet_bootstrap.cc:492] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:24.062453 3615 tablet_bootstrap.cc:654] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.063128 3640 tablet_bootstrap.cc:654] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.063773 3614 tablet_bootstrap.cc:654] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.063946 3615 tablet_bootstrap.cc:492] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:24.063984 3637 tablet_bootstrap.cc:492] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:24.064043 3615 ts_tablet_manager.cc:1367] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:24.064059 3637 ts_tablet_manager.cc:1367] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:24.064396 786 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b96ddb6ec9f4ef0bda0d354fc39adec with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:24.064438 3637 raft_consensus.cc:348] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.064482 3615 raft_consensus.cc:348] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.064568 3637 raft_consensus.cc:374] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.064584 3615 raft_consensus.cc:374] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.064608 3637 raft_consensus.cc:729] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.064616 3615 raft_consensus.cc:729] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.064730 3615 consensus_queue.cc:260] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.064816 3637 consensus_queue.cc:260] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.064976 3614 tablet_bootstrap.cc:492] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:24.065023 3615 ts_tablet_manager.cc:1398] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:24.065052 3614 ts_tablet_manager.cc:1367] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:24.065107 387 tablet_service.cc:1514] Processing DeleteTablet for tablet a3e7aa78b21a42f89cd3ac29cd113206 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:24.065253 3637 ts_tablet_manager.cc:1398] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:24.065402 386 tablet_service.cc:1514] Processing DeleteTablet for tablet 8122e08018cb40e987cdd4e9053c6cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:24.065538 388 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b96ddb6ec9f4ef0bda0d354fc39adec with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:24.065577 3637 tablet_bootstrap.cc:492] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:24.065518 3614 raft_consensus.cc:348] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:24.065624 3614 raft_consensus.cc:374] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.066012 3614 raft_consensus.cc:729] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.066228 3614 consensus_queue.cc:260] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:24.066617 3637 tablet_bootstrap.cc:654] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.067795 3495 tablet_bootstrap.cc:492] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:24.067909 3495 ts_tablet_manager.cc:1367] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240502 14:00:24.068383 3495 raft_consensus.cc:348] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:24.068514 3495 raft_consensus.cc:374] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.068564 3495 raft_consensus.cc:729] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.068688 3495 consensus_queue.cc:260] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:24.068953 3614 ts_tablet_manager.cc:1398] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:24.068986 3495 ts_tablet_manager.cc:1398] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:24.069110 3609 ts_tablet_manager.cc:1888] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.069201 3609 log.cc:1196] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/8122e08018cb40e987cdd4e9053c6cbd
I20240502 14:00:24.069499 3609 ts_tablet_manager.cc:1909] T 8122e08018cb40e987cdd4e9053c6cbd P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:24.069883 3643 tablet_replica.cc:330] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:24.070101 3643 raft_consensus.cc:2229] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.070327 3643 raft_consensus.cc:2258] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.070359 3648 tablet_replica.cc:330] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:24.070502 3648 raft_consensus.cc:2229] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.070657 3648 raft_consensus.cc:2258] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.070681 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 8122e08018cb40e987cdd4e9053c6cbd (table rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]) successfully deleted
I20240502 14:00:24.070878 3643 ts_tablet_manager.cc:1875] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.071138 3648 ts_tablet_manager.cc:1875] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.072530 3637 tablet_bootstrap.cc:492] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:24.072683 3637 ts_tablet_manager.cc:1367] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 14:00:24.072757 3640 tablet_bootstrap.cc:492] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:24.072939 3640 ts_tablet_manager.cc:1367] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 14:00:24.073276 3637 raft_consensus.cc:348] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.073345 3648 ts_tablet_manager.cc:1888] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.073377 3637 raft_consensus.cc:374] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.073411 3648 log.cc:1196] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/a3e7aa78b21a42f89cd3ac29cd113206
I20240502 14:00:24.073413 3637 raft_consensus.cc:729] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.073379 3640 raft_consensus.cc:348] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:24.073486 3640 raft_consensus.cc:374] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.073518 3640 raft_consensus.cc:729] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.073541 3637 consensus_queue.cc:260] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.073627 3640 consensus_queue.cc:260] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:24.073691 3648 ts_tablet_manager.cc:1909] T a3e7aa78b21a42f89cd3ac29cd113206 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:24.074326 3640 ts_tablet_manager.cc:1398] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:24.074437 3644 tablet_replica.cc:330] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:24.074474 3637 ts_tablet_manager.cc:1398] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:24.074543 3640 tablet_bootstrap.cc:492] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:24.074721 3644 raft_consensus.cc:2229] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.074971 3644 raft_consensus.cc:2258] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.075556 3644 ts_tablet_manager.cc:1875] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.075608 3640 tablet_bootstrap.cc:654] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.075858 3643 ts_tablet_manager.cc:1888] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.076001 3643 log.cc:1196] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/6b96ddb6ec9f4ef0bda0d354fc39adec
I20240502 14:00:24.076385 3643 ts_tablet_manager.cc:1909] T 6b96ddb6ec9f4ef0bda0d354fc39adec P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:24.076707 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet a3e7aa78b21a42f89cd3ac29cd113206 (table rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]) successfully deleted
I20240502 14:00:24.077057 3643 tablet_replica.cc:330] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:24.077988 3643 raft_consensus.cc:2229] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:24.078043 3648 tablet_replica.cc:330] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:24.078186 3643 raft_consensus.cc:2258] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.078186 3648 raft_consensus.cc:2229] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.078334 3648 raft_consensus.cc:2258] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.078724 3643 ts_tablet_manager.cc:1875] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.078895 3648 ts_tablet_manager.cc:1875] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.079185 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 6b96ddb6ec9f4ef0bda0d354fc39adec (table rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]) successfully deleted
I20240502 14:00:24.081514 3648 ts_tablet_manager.cc:1888] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.081737 3643 ts_tablet_manager.cc:1888] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.081813 3643 log.cc:1196] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/a3e7aa78b21a42f89cd3ac29cd113206
I20240502 14:00:24.082053 3643 ts_tablet_manager.cc:1909] T a3e7aa78b21a42f89cd3ac29cd113206 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:24.082268 3644 ts_tablet_manager.cc:1888] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.082340 3644 log.cc:1196] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/a3e7aa78b21a42f89cd3ac29cd113206
I20240502 14:00:24.082595 3644 ts_tablet_manager.cc:1909] T a3e7aa78b21a42f89cd3ac29cd113206 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:24.082818 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet a3e7aa78b21a42f89cd3ac29cd113206 (table rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]) successfully deleted
I20240502 14:00:24.083174 3644 tablet_replica.cc:330] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:24.083194 3640 tablet_bootstrap.cc:492] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:24.083281 3640 ts_tablet_manager.cc:1367] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 14:00:24.083284 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet a3e7aa78b21a42f89cd3ac29cd113206 (table rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]) successfully deleted
I20240502 14:00:24.083355 3644 raft_consensus.cc:2229] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.083483 3644 raft_consensus.cc:2258] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.083763 3640 raft_consensus.cc:348] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.083945 3644 ts_tablet_manager.cc:1875] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.083947 3640 raft_consensus.cc:374] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.083920 3648 log.cc:1196] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/8122e08018cb40e987cdd4e9053c6cbd
I20240502 14:00:24.084000 3640 raft_consensus.cc:729] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.084410 3640 consensus_queue.cc:260] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.084527 3648 ts_tablet_manager.cc:1909] T 8122e08018cb40e987cdd4e9053c6cbd P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:24.084779 3640 ts_tablet_manager.cc:1398] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:24.085569 3648 tablet_replica.cc:330] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:24.085727 3648 raft_consensus.cc:2229] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:24.085907 3648 raft_consensus.cc:2258] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.085997 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 8122e08018cb40e987cdd4e9053c6cbd (table rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]) successfully deleted
I20240502 14:00:24.086102 3644 ts_tablet_manager.cc:1888] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.086177 3644 log.cc:1196] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/8122e08018cb40e987cdd4e9053c6cbd
I20240502 14:00:24.086406 3644 ts_tablet_manager.cc:1909] T 8122e08018cb40e987cdd4e9053c6cbd P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:24.086717 3648 ts_tablet_manager.cc:1875] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.086926 3644 tablet_replica.cc:330] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:24.087121 3644 raft_consensus.cc:2229] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.087260 3644 raft_consensus.cc:2258] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.087445 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 8122e08018cb40e987cdd4e9053c6cbd (table rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]) successfully deleted
I20240502 14:00:24.087762 3644 ts_tablet_manager.cc:1875] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.088435 3648 ts_tablet_manager.cc:1888] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.088505 3648 log.cc:1196] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/6b96ddb6ec9f4ef0bda0d354fc39adec
I20240502 14:00:24.088719 3648 ts_tablet_manager.cc:1909] T 6b96ddb6ec9f4ef0bda0d354fc39adec P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:24.089313 3644 ts_tablet_manager.cc:1888] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.089381 3644 log.cc:1196] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/6b96ddb6ec9f4ef0bda0d354fc39adec
I20240502 14:00:24.089488 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 6b96ddb6ec9f4ef0bda0d354fc39adec (table rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]) successfully deleted
I20240502 14:00:24.089644 3644 ts_tablet_manager.cc:1909] T 6b96ddb6ec9f4ef0bda0d354fc39adec P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:24.090230 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 6b96ddb6ec9f4ef0bda0d354fc39adec (table rebalance_test_table_6 [id=ab84fa9b85094208a6c2e4a3b39c1cb2]) successfully deleted
I20240502 14:00:24.104192 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:24.109447 3578 tablet.cc:1722] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:24.110040 3627 tablet.cc:1722] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:24.110100 3583 tablet.cc:1722] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:24.115065 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=52e059b3549241d59ab5da3a91ab86d2] alter complete (version 3)
I20240502 14:00:24.217214 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:24.217391 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:24.219552 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet b379f016bee14bb29b6dd7033236a28a
I20240502 14:00:24.220183 520 tablet_service.cc:1514] Processing DeleteTablet for tablet b379f016bee14bb29b6dd7033236a28a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:24.220225 3262 tablet_service.cc:1514] Processing DeleteTablet for tablet b379f016bee14bb29b6dd7033236a28a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:24.220304 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet b379f016bee14bb29b6dd7033236a28a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:24.220475 3643 tablet_replica.cc:330] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:24.220595 3652 tablet_replica.cc:330] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:24.220681 3651 tablet_replica.cc:330] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:24.220746 3652 raft_consensus.cc:2229] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.220813 3651 raft_consensus.cc:2229] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:24.220887 3652 raft_consensus.cc:2258] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.221175 3651 raft_consensus.cc:2258] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.221694 3652 ts_tablet_manager.cc:1875] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.222044 3651 ts_tablet_manager.cc:1875] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.222204 3643 raft_consensus.cc:2229] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.222334 3643 raft_consensus.cc:2258] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.222918 3643 ts_tablet_manager.cc:1875] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.223768 3652 ts_tablet_manager.cc:1888] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:24.223866 3652 log.cc:1196] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/b379f016bee14bb29b6dd7033236a28a
I20240502 14:00:24.224152 3652 ts_tablet_manager.cc:1909] T b379f016bee14bb29b6dd7033236a28a P 4bcd7712aa844d98977a7d7e605c31f5: Deleting consensus metadata
I20240502 14:00:24.224617 3651 ts_tablet_manager.cc:1888] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:24.224692 3651 log.cc:1196] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/b379f016bee14bb29b6dd7033236a28a
I20240502 14:00:24.224932 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet b379f016bee14bb29b6dd7033236a28a (table rebalancer_dynamic_table_new_name [id=52e059b3549241d59ab5da3a91ab86d2]) successfully deleted
I20240502 14:00:24.225080 3651 ts_tablet_manager.cc:1909] T b379f016bee14bb29b6dd7033236a28a P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:24.225379 3643 ts_tablet_manager.cc:1888] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:24.225461 3643 log.cc:1196] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/b379f016bee14bb29b6dd7033236a28a
I20240502 14:00:24.225723 3643 ts_tablet_manager.cc:1909] T b379f016bee14bb29b6dd7033236a28a P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:24.226001 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet b379f016bee14bb29b6dd7033236a28a (table rebalancer_dynamic_table_new_name [id=52e059b3549241d59ab5da3a91ab86d2]) successfully deleted
I20240502 14:00:24.226498 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet b379f016bee14bb29b6dd7033236a28a (table rebalancer_dynamic_table_new_name [id=52e059b3549241d59ab5da3a91ab86d2]) successfully deleted
I20240502 14:00:24.237809 3581 raft_consensus.cc:482] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:24.237942 3581 raft_consensus.cc:504] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.238463 3581 leader_election.cc:288] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:24.243006 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ce30696a474d23818c3f70cc86e9dd" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:24.243188 3151 raft_consensus.cc:2454] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:24.243364 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ce30696a474d23818c3f70cc86e9dd" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:24.243518 3019 raft_consensus.cc:2454] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:24.243520 477 leader_election.cc:302] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:24.243973 3581 raft_consensus.cc:2781] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:24.244040 3581 raft_consensus.cc:482] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:24.244086 3581 raft_consensus.cc:3037] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.244935 3581 raft_consensus.cc:504] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.245316 3581 leader_election.cc:288] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:24.245631 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ce30696a474d23818c3f70cc86e9dd" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:24.245754 3019 raft_consensus.cc:3037] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.246136 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ce30696a474d23818c3f70cc86e9dd" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:24.246253 3151 raft_consensus.cc:3037] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.246804 3019 raft_consensus.cc:2454] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:24.247217 474 leader_election.cc:302] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:24.247309 3151 raft_consensus.cc:2454] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:24.247464 3581 raft_consensus.cc:2781] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:24.247560 3581 raft_consensus.cc:686] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:24.247706 3581 consensus_queue.cc:237] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.250098 32731 catalog_manager.cc:5484] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:24.273500 3503 raft_consensus.cc:482] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:24.273653 3503 raft_consensus.cc:504] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:24.274091 3503 leader_election.cc:288] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:24.274441 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d9d248c2c6745489d2656752db9ddd5" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:24.274577 3019 raft_consensus.cc:2454] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:24.274622 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d9d248c2c6745489d2656752db9ddd5" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:24.274809 3284 raft_consensus.cc:2454] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:24.274876 2819 leader_election.cc:302] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:24.275079 3503 raft_consensus.cc:2781] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:24.275141 3503 raft_consensus.cc:482] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:24.275197 3503 raft_consensus.cc:3037] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.276266 3503 raft_consensus.cc:504] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:24.276646 3503 leader_election.cc:288] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:24.276966 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d9d248c2c6745489d2656752db9ddd5" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:24.277109 3019 raft_consensus.cc:3037] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.277187 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d9d248c2c6745489d2656752db9ddd5" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:24.277288 3284 raft_consensus.cc:3037] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.277995 3019 raft_consensus.cc:2454] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:24.278074 3284 raft_consensus.cc:2454] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:24.278321 2820 leader_election.cc:302] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:24.278555 3503 raft_consensus.cc:2781] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:24.278645 3503 raft_consensus.cc:686] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:24.278789 3503 consensus_queue.cc:237] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:24.280683 32731 catalog_manager.cc:5484] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:24.321498 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:24.327188 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6262f5d8173e4887962e783a6ecd1252]), partition=
I20240502 14:00:24.327338 786 tablet_service.cc:1467] Processing CreateTablet for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6262f5d8173e4887962e783a6ecd1252]), partition=
I20240502 14:00:24.327373 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6262f5d8173e4887962e783a6ecd1252]), partition=
I20240502 14:00:24.327515 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.327572 786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.327620 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.329432 3495 tablet_bootstrap.cc:492] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:24.329771 3640 tablet_bootstrap.cc:492] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:24.329839 3659 tablet_bootstrap.cc:492] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:24.330371 3495 tablet_bootstrap.cc:654] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.330566 3640 tablet_bootstrap.cc:654] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.330705 3659 tablet_bootstrap.cc:654] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.331547 3495 tablet_bootstrap.cc:492] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:24.331640 3495 ts_tablet_manager.cc:1367] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:24.332127 3495 raft_consensus.cc:348] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:24.332237 3495 raft_consensus.cc:374] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.332288 3495 raft_consensus.cc:729] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.332420 3495 consensus_queue.cc:260] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:24.332727 3495 ts_tablet_manager.cc:1398] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:24.333382 3640 tablet_bootstrap.cc:492] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:24.333493 3640 ts_tablet_manager.cc:1367] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 14:00:24.333523 3659 tablet_bootstrap.cc:492] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:24.333614 3659 ts_tablet_manager.cc:1367] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:24.334053 3640 raft_consensus.cc:348] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:24.334055 3659 raft_consensus.cc:348] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:24.334154 3640 raft_consensus.cc:374] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.334170 3659 raft_consensus.cc:374] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.334188 3640 raft_consensus.cc:729] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.334211 3659 raft_consensus.cc:729] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.334318 3640 consensus_queue.cc:260] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:24.334348 3659 consensus_queue.cc:260] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:24.334564 3640 ts_tablet_manager.cc:1398] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 14:00:24.334656 3659 ts_tablet_manager.cc:1398] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:24.401449 3417 raft_consensus.cc:482] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:24.401562 3417 raft_consensus.cc:504] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:24.402145 3417 leader_election.cc:288] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:24.406512 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750fd0a66d2e4ce5b7f997a3fef7d4ad" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:24.406612 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750fd0a66d2e4ce5b7f997a3fef7d4ad" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:24.406682 2886 raft_consensus.cc:2454] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:24.406750 3019 raft_consensus.cc:2454] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:24.407033 738 leader_election.cc:302] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:24.407219 3417 raft_consensus.cc:2781] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:24.407300 3417 raft_consensus.cc:482] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:24.407330 3417 raft_consensus.cc:3037] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.408213 3417 raft_consensus.cc:504] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:24.408568 3417 leader_election.cc:288] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:24.408898 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750fd0a66d2e4ce5b7f997a3fef7d4ad" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:24.408913 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750fd0a66d2e4ce5b7f997a3fef7d4ad" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:24.409022 3019 raft_consensus.cc:3037] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.409022 2886 raft_consensus.cc:3037] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.409737 3019 raft_consensus.cc:2454] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:24.409796 2886 raft_consensus.cc:2454] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:24.410014 738 leader_election.cc:302] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:24.410252 3417 raft_consensus.cc:2781] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:24.410454 3417 raft_consensus.cc:686] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:24.410620 3417 consensus_queue.cc:237] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:24.412588 32732 catalog_manager.cc:5484] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:24.461894 3569 raft_consensus.cc:482] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:24.462018 3569 raft_consensus.cc:504] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.462446 3569 leader_election.cc:288] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:24.462867 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c02b136d2c492bb727e77793a1db20" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:24.463053 3284 raft_consensus.cc:2454] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:24.463392 3085 leader_election.cc:302] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:24.463586 3569 raft_consensus.cc:2781] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:24.463644 3569 raft_consensus.cc:482] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:24.463696 3569 raft_consensus.cc:3037] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.464740 3569 raft_consensus.cc:504] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.465143 3569 leader_election.cc:288] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:24.465521 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c02b136d2c492bb727e77793a1db20" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:24.465656 3284 raft_consensus.cc:3037] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.466446 3284 raft_consensus.cc:2454] T 90c02b136d2c492bb727e77793a1db20 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:24.466722 3085 leader_election.cc:302] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:24.467654 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c02b136d2c492bb727e77793a1db20" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:24.467850 540 raft_consensus.cc:2454] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:24.467664 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c02b136d2c492bb727e77793a1db20" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:24.467962 541 raft_consensus.cc:3037] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:24.468885 541 raft_consensus.cc:2454] T 90c02b136d2c492bb727e77793a1db20 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:24.473492 3569 raft_consensus.cc:2781] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:24.473678 3569 raft_consensus.cc:686] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:24.473837 3569 consensus_queue.cc:237] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:24.475708 32732 catalog_manager.cc:5484] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:24.531216 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:24.535781 2886 raft_consensus.cc:1261] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:24.535768 3019 raft_consensus.cc:1261] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:24.536266 3417 consensus_queue.cc:1035] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:24.536443 3667 consensus_queue.cc:1035] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:24.538870 32732 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad (table rebalancer_dynamic_table [id=6262f5d8173e4887962e783a6ecd1252]). Expected version 1 got 0
I20240502 14:00:24.539320 32731 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad (table rebalancer_dynamic_table [id=6262f5d8173e4887962e783a6ecd1252]). Expected version 1 got 0
I20240502 14:00:24.539700 3677 tablet.cc:1722] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:24.540560 3677 tablet.cc:1620] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:24.541008 3583 tablet.cc:1722] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:24.541311 3583 tablet.cc:1620] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:24.542181 32731 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad (table rebalancer_dynamic_table [id=6262f5d8173e4887962e783a6ecd1252]). Expected version 1 got 0
I20240502 14:00:24.542624 32730 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad (table rebalancer_dynamic_table [id=6262f5d8173e4887962e783a6ecd1252]). Expected version 1 got 0
I20240502 14:00:24.543053 3678 tablet.cc:1722] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:24.545998 3678 tablet.cc:1620] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:24.546909 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=6262f5d8173e4887962e783a6ecd1252] alter complete (version 1)
I20240502 14:00:24.562718 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_8" } modify_external_catalogs: true
I20240502 14:00:24.562875 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_8" } modify_external_catalogs: true
I20240502 14:00:24.565300 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet be4331e57df84e4ea7f92ff547cb6171
I20240502 14:00:24.565696 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet cc005fc5439b4ee6ac9beae8685a842d
I20240502 14:00:24.565884 388 tablet_service.cc:1514] Processing DeleteTablet for tablet be4331e57df84e4ea7f92ff547cb6171 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:24.565977 520 tablet_service.cc:1514] Processing DeleteTablet for tablet be4331e57df84e4ea7f92ff547cb6171 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:24.566088 3648 tablet_replica.cc:330] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:24.566063 653 tablet_service.cc:1514] Processing DeleteTablet for tablet be4331e57df84e4ea7f92ff547cb6171 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:24.566161 3643 tablet_replica.cc:330] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:24.566239 3609 tablet_replica.cc:330] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:24.566236 3648 raft_consensus.cc:2229] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:24.566299 3643 raft_consensus.cc:2229] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.566361 3609 raft_consensus.cc:2229] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.566432 3648 raft_consensus.cc:2258] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.566434 3643 raft_consensus.cc:2258] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.566488 3609 raft_consensus.cc:2258] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.566908 3648 ts_tablet_manager.cc:1875] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.566934 3609 ts_tablet_manager.cc:1875] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.566934 3643 ts_tablet_manager.cc:1875] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.568719 3648 ts_tablet_manager.cc:1888] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.568776 3648 log.cc:1196] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/be4331e57df84e4ea7f92ff547cb6171
I20240502 14:00:24.569043 3648 ts_tablet_manager.cc:1909] T be4331e57df84e4ea7f92ff547cb6171 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:24.569099 3609 ts_tablet_manager.cc:1888] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.569100 3643 ts_tablet_manager.cc:1888] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.569166 3643 log.cc:1196] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/be4331e57df84e4ea7f92ff547cb6171
I20240502 14:00:24.569166 3609 log.cc:1196] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/be4331e57df84e4ea7f92ff547cb6171
I20240502 14:00:24.569378 388 tablet_service.cc:1514] Processing DeleteTablet for tablet cc005fc5439b4ee6ac9beae8685a842d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:24.569437 3643 ts_tablet_manager.cc:1909] T be4331e57df84e4ea7f92ff547cb6171 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:24.569437 3609 ts_tablet_manager.cc:1909] T be4331e57df84e4ea7f92ff547cb6171 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:24.569799 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 457409778e844247a63aab5187494c74
I20240502 14:00:24.569780 786 tablet_service.cc:1514] Processing DeleteTablet for tablet cc005fc5439b4ee6ac9beae8685a842d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:24.570103 3644 tablet_replica.cc:330] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:24.570176 3648 tablet_replica.cc:330] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:24.570252 3644 raft_consensus.cc:2229] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:24.570287 3648 raft_consensus.cc:2229] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.570390 3648 raft_consensus.cc:2258] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.570443 3644 raft_consensus.cc:2258] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.570494 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet be4331e57df84e4ea7f92ff547cb6171 (table rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]) successfully deleted
I20240502 14:00:24.570852 3648 ts_tablet_manager.cc:1875] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.570899 3644 ts_tablet_manager.cc:1875] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.572067 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet be4331e57df84e4ea7f92ff547cb6171 (table rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]) successfully deleted
I20240502 14:00:24.572232 520 tablet_service.cc:1514] Processing DeleteTablet for tablet cc005fc5439b4ee6ac9beae8685a842d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:24.572232 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 457409778e844247a63aab5187494c74 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:24.572377 3643 tablet_replica.cc:330] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:24.572530 3643 raft_consensus.cc:2229] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.572607 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet be4331e57df84e4ea7f92ff547cb6171 (table rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]) successfully deleted
I20240502 14:00:24.572791 3643 raft_consensus.cc:2258] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.572907 388 tablet_service.cc:1514] Processing DeleteTablet for tablet 457409778e844247a63aab5187494c74 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:24.573163 786 tablet_service.cc:1514] Processing DeleteTablet for tablet 457409778e844247a63aab5187494c74 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:24.573287 3643 ts_tablet_manager.cc:1875] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.573529 3644 ts_tablet_manager.cc:1888] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.573529 3648 ts_tablet_manager.cc:1888] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.573603 3648 log.cc:1196] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/cc005fc5439b4ee6ac9beae8685a842d
I20240502 14:00:24.573603 3644 log.cc:1196] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/cc005fc5439b4ee6ac9beae8685a842d
I20240502 14:00:24.573874 3644 ts_tablet_manager.cc:1909] T cc005fc5439b4ee6ac9beae8685a842d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:24.573874 3648 ts_tablet_manager.cc:1909] T cc005fc5439b4ee6ac9beae8685a842d P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:24.573882 3609 tablet_replica.cc:330] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:24.574033 3609 raft_consensus.cc:2229] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:24.574491 3609 raft_consensus.cc:2258] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.574502 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet cc005fc5439b4ee6ac9beae8685a842d (table rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]) successfully deleted
I20240502 14:00:24.574553 3644 tablet_replica.cc:330] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:24.574878 3644 raft_consensus.cc:2229] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.574936 3648 tablet_replica.cc:330] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:24.575012 3644 raft_consensus.cc:2258] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.575055 3648 raft_consensus.cc:2229] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.575080 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet cc005fc5439b4ee6ac9beae8685a842d (table rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]) successfully deleted
I20240502 14:00:24.575181 3648 raft_consensus.cc:2258] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.575520 3609 ts_tablet_manager.cc:1875] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.575537 3644 ts_tablet_manager.cc:1875] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.575596 3648 ts_tablet_manager.cc:1875] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.576079 3643 ts_tablet_manager.cc:1888] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.576141 3643 log.cc:1196] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/cc005fc5439b4ee6ac9beae8685a842d
I20240502 14:00:24.576484 3643 ts_tablet_manager.cc:1909] T cc005fc5439b4ee6ac9beae8685a842d P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:24.577198 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet cc005fc5439b4ee6ac9beae8685a842d (table rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]) successfully deleted
I20240502 14:00:24.577662 3648 ts_tablet_manager.cc:1888] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.577744 3648 log.cc:1196] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/457409778e844247a63aab5187494c74
I20240502 14:00:24.578032 3648 ts_tablet_manager.cc:1909] T 457409778e844247a63aab5187494c74 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:24.578696 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 457409778e844247a63aab5187494c74 (table rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]) successfully deleted
I20240502 14:00:24.578899 3609 ts_tablet_manager.cc:1888] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.578981 3609 log.cc:1196] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/457409778e844247a63aab5187494c74
I20240502 14:00:24.579233 3609 ts_tablet_manager.cc:1909] T 457409778e844247a63aab5187494c74 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:24.579767 3644 ts_tablet_manager.cc:1888] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:24.579900 3644 log.cc:1196] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/457409778e844247a63aab5187494c74
I20240502 14:00:24.579890 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 457409778e844247a63aab5187494c74 (table rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]) successfully deleted
I20240502 14:00:24.580168 3644 ts_tablet_manager.cc:1909] T 457409778e844247a63aab5187494c74 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:24.580796 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 457409778e844247a63aab5187494c74 (table rebalance_test_table_8 [id=f0720f621f4d4dbc9b4f5a9c32260e6c]) successfully deleted
I20240502 14:00:24.644277 3579 consensus_queue.cc:1035] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:24.647632 3579 consensus_queue.cc:1035] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:24.650650 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:24.656342 3677 tablet.cc:1722] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:24.656688 3677 tablet.cc:1620] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:24.657001 3583 tablet.cc:1722] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:24.657229 3583 tablet.cc:1620] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:24.660411 3678 tablet.cc:1722] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:24.660741 3678 tablet.cc:1620] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:24.665477 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=6262f5d8173e4887962e783a6ecd1252] alter complete (version 2)
I20240502 14:00:24.667795 3503 consensus_queue.cc:1035] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:24.670796 3686 consensus_queue.cc:1035] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:24.768638 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:24.774080 3677 tablet.cc:1722] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:24.774634 3583 tablet.cc:1722] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:24.775305 3678 tablet.cc:1722] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:24.779670 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=6262f5d8173e4887962e783a6ecd1252] alter complete (version 3)
I20240502 14:00:24.881692 32730 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:24.881887 32730 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:24.883888 32730 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad
I20240502 14:00:24.884390 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:24.884428 786 tablet_service.cc:1514] Processing DeleteTablet for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:24.884613 3644 tablet_replica.cc:330] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:24.884624 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:24 UTC) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:24.884752 3691 tablet_replica.cc:330] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:24.884774 3644 raft_consensus.cc:2229] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:24.884881 3691 raft_consensus.cc:2229] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.884902 3692 tablet_replica.cc:330] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:24.884989 3691 raft_consensus.cc:2258] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.884995 3644 raft_consensus.cc:2258] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.885030 3692 raft_consensus.cc:2229] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:24.885141 3692 raft_consensus.cc:2258] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:24.885756 3691 ts_tablet_manager.cc:1875] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.885834 3644 ts_tablet_manager.cc:1875] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.886430 3692 ts_tablet_manager.cc:1875] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:24.887611 3691 ts_tablet_manager.cc:1888] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:24.887681 3691 log.cc:1196] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/750fd0a66d2e4ce5b7f997a3fef7d4ad
I20240502 14:00:24.888056 3644 ts_tablet_manager.cc:1888] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:24.888126 3644 log.cc:1196] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/750fd0a66d2e4ce5b7f997a3fef7d4ad
I20240502 14:00:24.888155 3691 ts_tablet_manager.cc:1909] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:24.888373 3644 ts_tablet_manager.cc:1909] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:24.888796 3692 ts_tablet_manager.cc:1888] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:24.888828 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad (table rebalancer_dynamic_table_new_name [id=6262f5d8173e4887962e783a6ecd1252]) successfully deleted
I20240502 14:00:24.888873 3692 log.cc:1196] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/750fd0a66d2e4ce5b7f997a3fef7d4ad
I20240502 14:00:24.889029 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad (table rebalancer_dynamic_table_new_name [id=6262f5d8173e4887962e783a6ecd1252]) successfully deleted
I20240502 14:00:24.889189 3692 ts_tablet_manager.cc:1909] T 750fd0a66d2e4ce5b7f997a3fef7d4ad P 4bcd7712aa844d98977a7d7e605c31f5: Deleting consensus metadata
I20240502 14:00:24.889789 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 750fd0a66d2e4ce5b7f997a3fef7d4ad (table rebalancer_dynamic_table_new_name [id=6262f5d8173e4887962e783a6ecd1252]) successfully deleted
I20240502 14:00:24.985766 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:24.990978 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 9b672a85a4784dd59c8f62a4351198aa (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f11aa7076a04442d9af086cbae95edbc]), partition=
I20240502 14:00:24.991071 3262 tablet_service.cc:1467] Processing CreateTablet for tablet 9b672a85a4784dd59c8f62a4351198aa (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f11aa7076a04442d9af086cbae95edbc]), partition=
I20240502 14:00:24.991143 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 9b672a85a4784dd59c8f62a4351198aa (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f11aa7076a04442d9af086cbae95edbc]), partition=
I20240502 14:00:24.991371 3262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b672a85a4784dd59c8f62a4351198aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.991371 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b672a85a4784dd59c8f62a4351198aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.991382 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b672a85a4784dd59c8f62a4351198aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:24.993443 3693 tablet_bootstrap.cc:492] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:24.993762 3695 tablet_bootstrap.cc:492] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:24.993794 3694 tablet_bootstrap.cc:492] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:24.994505 3694 tablet_bootstrap.cc:654] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.994720 3695 tablet_bootstrap.cc:654] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.994753 3693 tablet_bootstrap.cc:654] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:24.995549 3694 tablet_bootstrap.cc:492] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:24.995700 3694 ts_tablet_manager.cc:1367] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:24.995865 3695 tablet_bootstrap.cc:492] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:24.995909 3693 tablet_bootstrap.cc:492] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:24.996039 3695 ts_tablet_manager.cc:1367] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:24.996140 3693 ts_tablet_manager.cc:1367] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:24.996254 3694 raft_consensus.cc:348] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:24.996438 3694 raft_consensus.cc:374] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.996522 3694 raft_consensus.cc:729] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.996557 3695 raft_consensus.cc:348] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:24.996560 3693 raft_consensus.cc:348] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:24.996662 3693 raft_consensus.cc:374] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.996698 3695 raft_consensus.cc:374] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:24.996722 3693 raft_consensus.cc:729] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.996779 3695 raft_consensus.cc:729] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:24.996888 3693 consensus_queue.cc:260] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:24.996924 3695 consensus_queue.cc:260] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:24.997247 3695 ts_tablet_manager.cc:1398] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:24.997699 3693 ts_tablet_manager.cc:1398] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:24.997025 3694 consensus_queue.cc:260] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:24.998216 3694 ts_tablet_manager.cc:1398] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:25.015573 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:25.022646 2864 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc1880244a64b9f96ad5b8122a49fce (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.022763 2866 tablet_service.cc:1467] Processing CreateTablet for tablet e93a3a3fca9042e9b8bdf3c60ec73a20 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.022967 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc1880244a64b9f96ad5b8122a49fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.023162 2865 tablet_service.cc:1467] Processing CreateTablet for tablet f3d9ae2ed9a84c2c928dc1d03f325fa3 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.023368 2865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d9ae2ed9a84c2c928dc1d03f325fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.025583 3132 tablet_service.cc:1467] Processing CreateTablet for tablet f3d9ae2ed9a84c2c928dc1d03f325fa3 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.025843 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d9ae2ed9a84c2c928dc1d03f325fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.027863 3693 tablet_bootstrap.cc:492] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:25.028237 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93a3a3fca9042e9b8bdf3c60ec73a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.028560 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc1880244a64b9f96ad5b8122a49fce (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.028817 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc1880244a64b9f96ad5b8122a49fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.029081 2998 tablet_service.cc:1467] Processing CreateTablet for tablet f3d9ae2ed9a84c2c928dc1d03f325fa3 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.029341 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d9ae2ed9a84c2c928dc1d03f325fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.029698 3693 tablet_bootstrap.cc:654] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.030964 3694 tablet_bootstrap.cc:492] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:25.028555 3262 tablet_service.cc:1467] Processing CreateTablet for tablet e93a3a3fca9042e9b8bdf3c60ec73a20 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.030695 3699 tablet_bootstrap.cc:492] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:25.031252 3262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93a3a3fca9042e9b8bdf3c60ec73a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.031651 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc1880244a64b9f96ad5b8122a49fce (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.031908 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc1880244a64b9f96ad5b8122a49fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.032114 3694 tablet_bootstrap.cc:654] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.032176 3699 tablet_bootstrap.cc:654] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.033360 3699 tablet_bootstrap.cc:492] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:25.033496 3699 ts_tablet_manager.cc:1367] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:25.033943 3699 raft_consensus.cc:348] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.029048 2999 tablet_service.cc:1467] Processing CreateTablet for tablet e93a3a3fca9042e9b8bdf3c60ec73a20 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.034139 3699 raft_consensus.cc:374] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.034204 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93a3a3fca9042e9b8bdf3c60ec73a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.034229 3699 raft_consensus.cc:729] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.034368 3699 consensus_queue.cc:260] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.034657 3699 ts_tablet_manager.cc:1398] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:25.034901 3699 tablet_bootstrap.cc:492] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:25.036144 3693 tablet_bootstrap.cc:492] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:25.036249 3693 ts_tablet_manager.cc:1367] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:25.036706 3693 raft_consensus.cc:348] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.036823 3693 raft_consensus.cc:374] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.036870 3693 raft_consensus.cc:729] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.037001 3693 consensus_queue.cc:260] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.037250 3693 ts_tablet_manager.cc:1398] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:25.038822 3695 tablet_bootstrap.cc:492] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:25.039902 3695 tablet_bootstrap.cc:654] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.041150 3695 tablet_bootstrap.cc:492] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:25.041292 3695 ts_tablet_manager.cc:1367] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:25.042152 3699 tablet_bootstrap.cc:654] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.042289 3695 raft_consensus.cc:348] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.042387 3695 raft_consensus.cc:374] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.042421 3695 raft_consensus.cc:729] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.042565 3695 consensus_queue.cc:260] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.042845 3695 ts_tablet_manager.cc:1398] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:25.043134 3695 tablet_bootstrap.cc:492] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:25.043905 3695 tablet_bootstrap.cc:654] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.044549 3694 tablet_bootstrap.cc:492] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:25.044659 3694 ts_tablet_manager.cc:1367] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 14:00:25.045174 3695 tablet_bootstrap.cc:492] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:25.045161 3694 raft_consensus.cc:348] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:25.045246 3695 ts_tablet_manager.cc:1367] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:25.045277 3694 raft_consensus.cc:374] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.045310 3694 raft_consensus.cc:729] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.045452 3694 consensus_queue.cc:260] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:25.045655 3695 raft_consensus.cc:348] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:25.045739 3695 raft_consensus.cc:374] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.045738 3694 ts_tablet_manager.cc:1398] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:25.045773 3695 raft_consensus.cc:729] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.045928 3694 tablet_bootstrap.cc:492] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:25.045907 3695 consensus_queue.cc:260] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:25.046146 3695 ts_tablet_manager.cc:1398] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:25.046301 3695 tablet_bootstrap.cc:492] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:25.046528 3699 tablet_bootstrap.cc:492] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:25.046607 3699 ts_tablet_manager.cc:1367] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 14:00:25.046867 3694 tablet_bootstrap.cc:654] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.047003 3699 raft_consensus.cc:348] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:25.047144 3699 raft_consensus.cc:374] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.047173 3699 raft_consensus.cc:729] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.047299 3695 tablet_bootstrap.cc:654] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.047324 3699 consensus_queue.cc:260] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:25.049080 3695 tablet_bootstrap.cc:492] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:25.049156 3695 ts_tablet_manager.cc:1367] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:25.049512 3695 raft_consensus.cc:348] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.049597 3695 raft_consensus.cc:374] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.049625 3695 raft_consensus.cc:729] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.049733 3695 consensus_queue.cc:260] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.049988 3695 ts_tablet_manager.cc:1398] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:25.052172 3699 ts_tablet_manager.cc:1398] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:00:25.052381 3699 tablet_bootstrap.cc:492] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:25.053661 3699 tablet_bootstrap.cc:654] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.054922 3699 tablet_bootstrap.cc:492] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:25.055054 3694 tablet_bootstrap.cc:492] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:25.055141 3694 ts_tablet_manager.cc:1367] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:25.055227 3699 ts_tablet_manager.cc:1367] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:25.055608 3694 raft_consensus.cc:348] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.055742 3694 raft_consensus.cc:374] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.055784 3694 raft_consensus.cc:729] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.055945 3694 consensus_queue.cc:260] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.055999 3699 raft_consensus.cc:348] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.056236 3694 ts_tablet_manager.cc:1398] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:25.056259 3699 raft_consensus.cc:374] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.056396 3699 raft_consensus.cc:729] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.056607 3699 consensus_queue.cc:260] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.056950 3699 ts_tablet_manager.cc:1398] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:25.073294 32732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_10" } modify_external_catalogs: true
I20240502 14:00:25.073338 3569 consensus_queue.cc:1035] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.073448 32732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_10" } modify_external_catalogs: true
I20240502 14:00:25.076040 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 8f862cbcf8fc49598ef9cfbca658fc5b
I20240502 14:00:25.076661 786 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f862cbcf8fc49598ef9cfbca658fc5b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:25.076850 3644 tablet_replica.cc:330] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:25.076994 3644 raft_consensus.cc:2229] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.077147 3644 raft_consensus.cc:2258] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.077713 3644 ts_tablet_manager.cc:1875] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.079671 388 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f862cbcf8fc49598ef9cfbca658fc5b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:25.079771 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 9ebc7252d1534e31b23b79c806416bee
I20240502 14:00:25.079957 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f862cbcf8fc49598ef9cfbca658fc5b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:25.080255 388 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ebc7252d1534e31b23b79c806416bee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:25.080881 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet efe3db917ec341dc935709b8e66e969b
I20240502 14:00:25.081148 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ebc7252d1534e31b23b79c806416bee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:25.081430 786 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ebc7252d1534e31b23b79c806416bee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:25.081512 784 tablet_service.cc:1514] Processing DeleteTablet for tablet efe3db917ec341dc935709b8e66e969b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:25.081617 388 tablet_service.cc:1514] Processing DeleteTablet for tablet efe3db917ec341dc935709b8e66e969b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:25.081609 3711 tablet_replica.cc:330] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:25.081887 3711 raft_consensus.cc:2229] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:25.082132 3711 raft_consensus.cc:2258] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.082301 520 tablet_service.cc:1514] Processing DeleteTablet for tablet efe3db917ec341dc935709b8e66e969b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:25.082547 3713 tablet_replica.cc:330] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:25.082710 3713 raft_consensus.cc:2229] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:25.082921 3713 raft_consensus.cc:2258] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.083511 3713 ts_tablet_manager.cc:1875] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.083628 3711 ts_tablet_manager.cc:1875] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.083706 3644 ts_tablet_manager.cc:1888] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.083788 3644 log.cc:1196] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/8f862cbcf8fc49598ef9cfbca658fc5b
I20240502 14:00:25.084108 3644 ts_tablet_manager.cc:1909] T 8f862cbcf8fc49598ef9cfbca658fc5b P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:25.085665 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 8f862cbcf8fc49598ef9cfbca658fc5b (table rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]) successfully deleted
I20240502 14:00:25.085741 3644 tablet_replica.cc:330] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:25.085888 3644 raft_consensus.cc:2229] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.085920 3713 ts_tablet_manager.cc:1888] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.085984 3713 log.cc:1196] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/9ebc7252d1534e31b23b79c806416bee
I20240502 14:00:25.086161 3644 raft_consensus.cc:2258] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.086254 3713 ts_tablet_manager.cc:1909] T 9ebc7252d1534e31b23b79c806416bee P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:25.086627 3711 ts_tablet_manager.cc:1888] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.086696 3711 log.cc:1196] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/8f862cbcf8fc49598ef9cfbca658fc5b
I20240502 14:00:25.086915 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 9ebc7252d1534e31b23b79c806416bee (table rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]) successfully deleted
I20240502 14:00:25.086989 3644 ts_tablet_manager.cc:1875] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.087033 3711 ts_tablet_manager.cc:1909] T 8f862cbcf8fc49598ef9cfbca658fc5b P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:25.087534 3711 tablet_replica.cc:330] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:25.087641 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 8f862cbcf8fc49598ef9cfbca658fc5b (table rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]) successfully deleted
I20240502 14:00:25.087674 3711 raft_consensus.cc:2229] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.087792 3711 raft_consensus.cc:2258] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.088302 3711 ts_tablet_manager.cc:1875] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.088788 3644 ts_tablet_manager.cc:1888] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.088856 3644 log.cc:1196] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/9ebc7252d1534e31b23b79c806416bee
I20240502 14:00:25.089084 3644 ts_tablet_manager.cc:1909] T 9ebc7252d1534e31b23b79c806416bee P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:25.089547 3712 tablet_replica.cc:330] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:25.089689 3644 tablet_replica.cc:330] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:25.089840 3644 raft_consensus.cc:2229] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:25.089857 3712 raft_consensus.cc:2229] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.090049 3712 raft_consensus.cc:2258] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.090057 3644 raft_consensus.cc:2258] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.090147 3711 ts_tablet_manager.cc:1888] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.090274 3711 log.cc:1196] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/9ebc7252d1534e31b23b79c806416bee
I20240502 14:00:25.090346 3710 consensus_queue.cc:1035] T 90c02b136d2c492bb727e77793a1db20 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.090411 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 9ebc7252d1534e31b23b79c806416bee (table rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]) successfully deleted
I20240502 14:00:25.090516 3644 ts_tablet_manager.cc:1875] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.090610 3711 ts_tablet_manager.cc:1909] T 9ebc7252d1534e31b23b79c806416bee P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:25.091037 3712 ts_tablet_manager.cc:1875] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.091203 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 9ebc7252d1534e31b23b79c806416bee (table rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]) successfully deleted
I20240502 14:00:25.091236 3711 tablet_replica.cc:330] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:25.091343 3711 raft_consensus.cc:2229] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.091518 3711 raft_consensus.cc:2258] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.092864 3711 ts_tablet_manager.cc:1875] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.093726 3712 ts_tablet_manager.cc:1888] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.093863 3712 log.cc:1196] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/8f862cbcf8fc49598ef9cfbca658fc5b
I20240502 14:00:25.094218 3712 ts_tablet_manager.cc:1909] T 8f862cbcf8fc49598ef9cfbca658fc5b P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:25.094497 3644 ts_tablet_manager.cc:1888] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.094578 3644 log.cc:1196] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/efe3db917ec341dc935709b8e66e969b
I20240502 14:00:25.094658 3711 ts_tablet_manager.cc:1888] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.094740 3711 log.cc:1196] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/efe3db917ec341dc935709b8e66e969b
I20240502 14:00:25.094864 3644 ts_tablet_manager.cc:1909] T efe3db917ec341dc935709b8e66e969b P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:25.094993 3712 tablet_replica.cc:330] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:25.095044 3711 ts_tablet_manager.cc:1909] T efe3db917ec341dc935709b8e66e969b P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:25.095088 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 8f862cbcf8fc49598ef9cfbca658fc5b (table rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]) successfully deleted
I20240502 14:00:25.095191 3712 raft_consensus.cc:2229] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.095331 3712 raft_consensus.cc:2258] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.095571 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet efe3db917ec341dc935709b8e66e969b (table rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]) successfully deleted
I20240502 14:00:25.095724 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet efe3db917ec341dc935709b8e66e969b (table rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]) successfully deleted
I20240502 14:00:25.096074 3712 ts_tablet_manager.cc:1875] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.097644 3712 ts_tablet_manager.cc:1888] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.097707 3712 log.cc:1196] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/efe3db917ec341dc935709b8e66e969b
I20240502 14:00:25.097923 3712 ts_tablet_manager.cc:1909] T efe3db917ec341dc935709b8e66e969b P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:25.098619 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet efe3db917ec341dc935709b8e66e969b (table rebalance_test_table_10 [id=7c3047e09b274700a6b47dd482142df0]) successfully deleted
I20240502 14:00:25.122754 3686 raft_consensus.cc:482] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:25.122903 3686 raft_consensus.cc:504] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:25.123378 3686 leader_election.cc:288] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:25.123757 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc1880244a64b9f96ad5b8122a49fce" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:25.123773 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc1880244a64b9f96ad5b8122a49fce" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:25.123931 3284 raft_consensus.cc:2454] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:25.123986 3019 raft_consensus.cc:2454] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:25.124218 2820 leader_election.cc:302] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:25.124392 3686 raft_consensus.cc:2781] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:25.124452 3686 raft_consensus.cc:482] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:25.124480 3686 raft_consensus.cc:3037] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.125320 3686 raft_consensus.cc:504] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:25.125715 3686 leader_election.cc:288] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:25.125814 3686 raft_consensus.cc:482] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:25.125873 3686 raft_consensus.cc:504] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.126024 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc1880244a64b9f96ad5b8122a49fce" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:25.126091 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc1880244a64b9f96ad5b8122a49fce" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:25.126139 3019 raft_consensus.cc:3037] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.126197 3284 raft_consensus.cc:3037] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.126291 3686 leader_election.cc:288] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:25.126710 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d9ae2ed9a84c2c928dc1d03f325fa3" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:25.126845 3018 raft_consensus.cc:2454] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:25.127089 3284 raft_consensus.cc:2454] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:25.127089 3019 raft_consensus.cc:2454] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:25.127149 2819 leader_election.cc:302] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:25.127295 3686 raft_consensus.cc:2781] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:25.127344 3686 raft_consensus.cc:482] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:25.127393 3686 raft_consensus.cc:3037] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.127427 2819 leader_election.cc:302] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:25.127580 3503 raft_consensus.cc:2781] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:25.127676 3503 raft_consensus.cc:686] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:25.127815 3503 consensus_queue.cc:237] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:25.128666 3686 raft_consensus.cc:504] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.129426 3686 leader_election.cc:288] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:25.129715 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d9ae2ed9a84c2c928dc1d03f325fa3" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:25.129860 3019 raft_consensus.cc:3037] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.129770 32732 catalog_manager.cc:5484] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:25.130714 3019 raft_consensus.cc:2454] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:25.131018 2819 leader_election.cc:302] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:25.131206 3686 raft_consensus.cc:2781] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:25.131341 3686 raft_consensus.cc:686] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:25.131469 3686 consensus_queue.cc:237] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.132063 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d9ae2ed9a84c2c928dc1d03f325fa3" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:25.132063 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d9ae2ed9a84c2c928dc1d03f325fa3" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:25.132226 3151 raft_consensus.cc:2454] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:25.133543 32732 catalog_manager.cc:5484] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:25.169505 3663 raft_consensus.cc:482] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:25.169658 3663 raft_consensus.cc:504] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.170140 3663 leader_election.cc:288] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:25.170758 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b672a85a4784dd59c8f62a4351198aa" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:25.170914 3152 raft_consensus.cc:2454] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:25.171232 2954 leader_election.cc:302] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:25.171427 3663 raft_consensus.cc:2781] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:25.171497 3663 raft_consensus.cc:482] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:25.171545 3663 raft_consensus.cc:3037] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.172654 3663 raft_consensus.cc:504] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.173017 3663 leader_election.cc:288] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:25.173352 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b672a85a4784dd59c8f62a4351198aa" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:25.173467 3152 raft_consensus.cc:3037] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.174469 3152 raft_consensus.cc:2454] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:25.174628 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b672a85a4784dd59c8f62a4351198aa" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:25.174790 2954 leader_election.cc:302] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:25.174651 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b672a85a4784dd59c8f62a4351198aa" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:25.174858 3284 raft_consensus.cc:2454] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:25.174954 3663 raft_consensus.cc:2781] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:25.175166 3663 raft_consensus.cc:686] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:25.175318 3663 consensus_queue.cc:237] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.177039 32732 catalog_manager.cc:5484] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:25.291113 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:25.294942 3284 raft_consensus.cc:3037] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.295210 3152 raft_consensus.cc:1261] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:25.295583 3663 consensus_queue.cc:1035] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.296103 3284 raft_consensus.cc:1261] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:25.296456 3663 consensus_queue.cc:1035] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.298010 32732 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 9b672a85a4784dd59c8f62a4351198aa (table rebalancer_dynamic_table [id=f11aa7076a04442d9af086cbae95edbc]). Expected version 1 got 0
I20240502 14:00:25.298802 32733 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 9b672a85a4784dd59c8f62a4351198aa (table rebalancer_dynamic_table [id=f11aa7076a04442d9af086cbae95edbc]). Expected version 1 got 0
I20240502 14:00:25.299391 3729 tablet.cc:1722] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:25.299779 3729 tablet.cc:1620] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:25.301118 32733 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 9b672a85a4784dd59c8f62a4351198aa (table rebalancer_dynamic_table [id=f11aa7076a04442d9af086cbae95edbc]). Expected version 1 got 0
I20240502 14:00:25.301213 3731 tablet.cc:1722] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:25.301527 3731 tablet.cc:1620] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:25.301755 32732 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet 9b672a85a4784dd59c8f62a4351198aa (table rebalancer_dynamic_table [id=f11aa7076a04442d9af086cbae95edbc]). Expected version 1 got 0
I20240502 14:00:25.302954 32733 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 9b672a85a4784dd59c8f62a4351198aa (table rebalancer_dynamic_table [id=f11aa7076a04442d9af086cbae95edbc]). Expected version 1 got 0
I20240502 14:00:25.305028 3730 tablet.cc:1722] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:25.305341 3730 tablet.cc:1620] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:25.306094 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=f11aa7076a04442d9af086cbae95edbc] alter complete (version 1)
I20240502 14:00:25.315132 3663 raft_consensus.cc:482] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:25.315233 3663 raft_consensus.cc:504] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.315639 3663 leader_election.cc:288] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:25.316015 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93a3a3fca9042e9b8bdf3c60ec73a20" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:25.316154 3284 raft_consensus.cc:2454] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:25.316421 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93a3a3fca9042e9b8bdf3c60ec73a20" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:25.316462 2952 leader_election.cc:302] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:25.316547 2886 raft_consensus.cc:2454] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:25.316632 3663 raft_consensus.cc:2781] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:25.316679 3663 raft_consensus.cc:482] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:25.316717 3663 raft_consensus.cc:3037] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.317616 3663 raft_consensus.cc:504] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.317916 3663 leader_election.cc:288] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:25.318375 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93a3a3fca9042e9b8bdf3c60ec73a20" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:25.318418 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93a3a3fca9042e9b8bdf3c60ec73a20" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:25.318511 3284 raft_consensus.cc:3037] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.318516 2886 raft_consensus.cc:3037] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.319393 2886 raft_consensus.cc:2454] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:25.319393 3284 raft_consensus.cc:2454] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:25.319671 2952 leader_election.cc:302] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:25.319911 3663 raft_consensus.cc:2781] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:25.319985 3663 raft_consensus.cc:686] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:25.320122 3663 consensus_queue.cc:237] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:25.321775 32732 catalog_manager.cc:5484] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:25.409493 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:25.415678 3731 tablet.cc:1722] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:25.416026 3731 tablet.cc:1620] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:25.416117 3730 tablet.cc:1722] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:25.416604 3730 tablet.cc:1620] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:25.418888 3729 tablet.cc:1722] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:25.419183 3729 tablet.cc:1620] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:25.420665 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=f11aa7076a04442d9af086cbae95edbc] alter complete (version 2)
I20240502 14:00:25.523275 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:25.529017 3686 consensus_queue.cc:1035] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.529651 3731 tablet.cc:1722] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:25.530323 3730 tablet.cc:1722] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:25.533119 3152 raft_consensus.cc:3037] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.534590 3686 consensus_queue.cc:1035] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.534935 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=f11aa7076a04442d9af086cbae95edbc] alter complete (version 3)
I20240502 14:00:25.537979 3729 tablet.cc:1722] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:25.583281 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_12" } modify_external_catalogs: true
I20240502 14:00:25.583436 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_12" } modify_external_catalogs: true
I20240502 14:00:25.586293 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 95cf29cac53140ac8ab67ad52b95a886
I20240502 14:00:25.586606 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet fb75f88d4c7e42f787118144e9f6c430
I20240502 14:00:25.586890 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 95cf29cac53140ac8ab67ad52b95a886 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:25.586961 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 95cf29cac53140ac8ab67ad52b95a886 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:25.587134 3713 tablet_replica.cc:330] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:25.587293 3713 raft_consensus.cc:2229] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:25.587496 3713 raft_consensus.cc:2258] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.587556 784 tablet_service.cc:1514] Processing DeleteTablet for tablet fb75f88d4c7e42f787118144e9f6c430 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:25.587720 3644 tablet_replica.cc:330] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:25.587872 3644 raft_consensus.cc:2229] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.587939 3712 tablet_replica.cc:330] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:25.588006 3644 raft_consensus.cc:2258] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.588071 3712 raft_consensus.cc:2229] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.588201 3712 raft_consensus.cc:2258] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.588537 3644 ts_tablet_manager.cc:1875] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.588649 3712 ts_tablet_manager.cc:1875] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.590729 3712 ts_tablet_manager.cc:1888] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.590729 3644 ts_tablet_manager.cc:1888] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.590793 3644 log.cc:1196] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/fb75f88d4c7e42f787118144e9f6c430
I20240502 14:00:25.590793 3712 log.cc:1196] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/95cf29cac53140ac8ab67ad52b95a886
I20240502 14:00:25.591068 3644 ts_tablet_manager.cc:1909] T fb75f88d4c7e42f787118144e9f6c430 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:25.591068 3712 ts_tablet_manager.cc:1909] T 95cf29cac53140ac8ab67ad52b95a886 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:25.591254 520 tablet_service.cc:1514] Processing DeleteTablet for tablet fb75f88d4c7e42f787118144e9f6c430 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:25.591545 388 tablet_service.cc:1514] Processing DeleteTablet for tablet 95cf29cac53140ac8ab67ad52b95a886 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:25.591727 3711 tablet_replica.cc:330] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:25.591883 3711 raft_consensus.cc:2229] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.592022 3711 raft_consensus.cc:2258] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.592514 3711 ts_tablet_manager.cc:1875] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.592983 3712 tablet_replica.cc:330] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:25.593096 3712 raft_consensus.cc:2229] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.593199 3712 raft_consensus.cc:2258] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.593696 3712 ts_tablet_manager.cc:1875] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.594650 3711 ts_tablet_manager.cc:1888] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.594717 3711 log.cc:1196] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/95cf29cac53140ac8ab67ad52b95a886
I20240502 14:00:25.595014 3711 ts_tablet_manager.cc:1909] T 95cf29cac53140ac8ab67ad52b95a886 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:25.595629 3712 ts_tablet_manager.cc:1888] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.595708 3712 log.cc:1196] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/fb75f88d4c7e42f787118144e9f6c430
I20240502 14:00:25.595702 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 95cf29cac53140ac8ab67ad52b95a886 (table rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]) successfully deleted
I20240502 14:00:25.595929 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 95cf29cac53140ac8ab67ad52b95a886 (table rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]) successfully deleted
I20240502 14:00:25.596001 3712 ts_tablet_manager.cc:1909] T fb75f88d4c7e42f787118144e9f6c430 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:25.596328 3713 ts_tablet_manager.cc:1875] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.596781 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet fb75f88d4c7e42f787118144e9f6c430 (table rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]) successfully deleted
I20240502 14:00:25.596901 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet fb75f88d4c7e42f787118144e9f6c430 (table rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]) successfully deleted
I20240502 14:00:25.597151 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 34e6f7677fbe40568b51a023be4c1ccd
I20240502 14:00:25.597491 388 tablet_service.cc:1514] Processing DeleteTablet for tablet fb75f88d4c7e42f787118144e9f6c430 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:25.597769 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 34e6f7677fbe40568b51a023be4c1ccd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:25.597947 3644 tablet_replica.cc:330] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:25.598067 3644 raft_consensus.cc:2229] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:25.598232 3644 raft_consensus.cc:2258] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.598466 3713 ts_tablet_manager.cc:1888] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.598467 388 tablet_service.cc:1514] Processing DeleteTablet for tablet 34e6f7677fbe40568b51a023be4c1ccd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:25.598534 3713 log.cc:1196] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/95cf29cac53140ac8ab67ad52b95a886
I20240502 14:00:25.598696 3644 ts_tablet_manager.cc:1875] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.598745 3713 ts_tablet_manager.cc:1909] T 95cf29cac53140ac8ab67ad52b95a886 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:25.599117 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 34e6f7677fbe40568b51a023be4c1ccd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:25.599445 3713 tablet_replica.cc:330] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:25.599592 3713 raft_consensus.cc:2229] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.599730 3713 raft_consensus.cc:2258] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.600220 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 95cf29cac53140ac8ab67ad52b95a886 (table rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]) successfully deleted
I20240502 14:00:25.600220 3713 ts_tablet_manager.cc:1875] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.600610 3644 ts_tablet_manager.cc:1888] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.600706 3644 log.cc:1196] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/34e6f7677fbe40568b51a023be4c1ccd
I20240502 14:00:25.600968 3644 ts_tablet_manager.cc:1909] T 34e6f7677fbe40568b51a023be4c1ccd P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:25.601104 3711 tablet_replica.cc:330] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:25.601245 3711 raft_consensus.cc:2229] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:25.601461 3686 consensus_queue.cc:1035] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.601780 3711 raft_consensus.cc:2258] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.602279 3711 ts_tablet_manager.cc:1875] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.604197 3713 ts_tablet_manager.cc:1888] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.604283 3713 log.cc:1196] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/34e6f7677fbe40568b51a023be4c1ccd
I20240502 14:00:25.604616 3713 ts_tablet_manager.cc:1909] T 34e6f7677fbe40568b51a023be4c1ccd P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:25.605366 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 34e6f7677fbe40568b51a023be4c1ccd (table rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]) successfully deleted
I20240502 14:00:25.605427 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 34e6f7677fbe40568b51a023be4c1ccd (table rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]) successfully deleted
I20240502 14:00:25.606123 3711 ts_tablet_manager.cc:1888] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.606254 3711 log.cc:1196] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/fb75f88d4c7e42f787118144e9f6c430
I20240502 14:00:25.606629 3711 ts_tablet_manager.cc:1909] T fb75f88d4c7e42f787118144e9f6c430 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:25.607015 3509 consensus_queue.cc:1035] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.607345 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet fb75f88d4c7e42f787118144e9f6c430 (table rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]) successfully deleted
I20240502 14:00:25.607493 3711 tablet_replica.cc:330] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:25.607647 3711 raft_consensus.cc:2229] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.607869 3711 raft_consensus.cc:2258] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.608628 3711 ts_tablet_manager.cc:1875] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.610527 3711 ts_tablet_manager.cc:1888] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:25.610586 3711 log.cc:1196] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/34e6f7677fbe40568b51a023be4c1ccd
I20240502 14:00:25.610792 3711 ts_tablet_manager.cc:1909] T 34e6f7677fbe40568b51a023be4c1ccd P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:25.611438 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 34e6f7677fbe40568b51a023be4c1ccd (table rebalance_test_table_12 [id=aca83c4d60264b2890f66a8eece98df5]) successfully deleted
I20240502 14:00:25.636633 32733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:25.636804 32733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:25.638790 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 9b672a85a4784dd59c8f62a4351198aa
I20240502 14:00:25.639370 3262 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b672a85a4784dd59c8f62a4351198aa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:25.639392 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b672a85a4784dd59c8f62a4351198aa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:25.639425 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b672a85a4784dd59c8f62a4351198aa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:25 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:25.639765 3741 tablet_replica.cc:330] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:25.640026 3741 raft_consensus.cc:2229] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:25.640048 3740 tablet_replica.cc:330] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:25.640183 3740 raft_consensus.cc:2229] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.640264 3741 raft_consensus.cc:2258] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.640319 3740 raft_consensus.cc:2258] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.640427 3739 tablet_replica.cc:330] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:25.640609 3739 raft_consensus.cc:2229] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:25.640763 3739 raft_consensus.cc:2258] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:25.640841 3741 ts_tablet_manager.cc:1875] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.641032 3740 ts_tablet_manager.cc:1875] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.641327 3739 ts_tablet_manager.cc:1875] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:25.642910 3741 ts_tablet_manager.cc:1888] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:25.643021 3741 log.cc:1196] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/9b672a85a4784dd59c8f62a4351198aa
I20240502 14:00:25.643290 3740 ts_tablet_manager.cc:1888] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:25.643350 3741 ts_tablet_manager.cc:1909] T 9b672a85a4784dd59c8f62a4351198aa P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:25.643393 3740 log.cc:1196] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/9b672a85a4784dd59c8f62a4351198aa
I20240502 14:00:25.643669 3739 ts_tablet_manager.cc:1888] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:25.643752 3739 log.cc:1196] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/9b672a85a4784dd59c8f62a4351198aa
I20240502 14:00:25.643770 3740 ts_tablet_manager.cc:1909] T 9b672a85a4784dd59c8f62a4351198aa P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:25.644052 3739 ts_tablet_manager.cc:1909] T 9b672a85a4784dd59c8f62a4351198aa P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:25.644050 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 9b672a85a4784dd59c8f62a4351198aa (table rebalancer_dynamic_table_new_name [id=f11aa7076a04442d9af086cbae95edbc]) successfully deleted
I20240502 14:00:25.644521 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 9b672a85a4784dd59c8f62a4351198aa (table rebalancer_dynamic_table_new_name [id=f11aa7076a04442d9af086cbae95edbc]) successfully deleted
I20240502 14:00:25.644701 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 9b672a85a4784dd59c8f62a4351198aa (table rebalancer_dynamic_table_new_name [id=f11aa7076a04442d9af086cbae95edbc]) successfully deleted
I20240502 14:00:25.741041 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:25.746081 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 685e3ae151f9458f83b303061a5a4b84 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9e7442eda1d343028feacdcecaeda624]), partition=
I20240502 14:00:25.746344 653 tablet_service.cc:1467] Processing CreateTablet for tablet 685e3ae151f9458f83b303061a5a4b84 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9e7442eda1d343028feacdcecaeda624]), partition=
I20240502 14:00:25.746429 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685e3ae151f9458f83b303061a5a4b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.746533 388 tablet_service.cc:1467] Processing CreateTablet for tablet 685e3ae151f9458f83b303061a5a4b84 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9e7442eda1d343028feacdcecaeda624]), partition=
I20240502 14:00:25.746619 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685e3ae151f9458f83b303061a5a4b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.746752 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685e3ae151f9458f83b303061a5a4b84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.748646 3742 tablet_bootstrap.cc:492] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:25.749063 3743 tablet_bootstrap.cc:492] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:25.749222 3744 tablet_bootstrap.cc:492] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:25.749861 3742 tablet_bootstrap.cc:654] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.749948 3743 tablet_bootstrap.cc:654] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.749974 3744 tablet_bootstrap.cc:654] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.750946 3743 tablet_bootstrap.cc:492] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:25.750946 3744 tablet_bootstrap.cc:492] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:25.751039 3743 ts_tablet_manager.cc:1367] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:25.751039 3744 ts_tablet_manager.cc:1367] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:25.751436 3743 raft_consensus.cc:348] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:25.751436 3744 raft_consensus.cc:348] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:25.751559 3744 raft_consensus.cc:374] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.751559 3743 raft_consensus.cc:374] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.751595 3744 raft_consensus.cc:729] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.751595 3743 raft_consensus.cc:729] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.751714 3743 consensus_queue.cc:260] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:25.751714 3744 consensus_queue.cc:260] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:25.752009 3743 ts_tablet_manager.cc:1398] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:25.752017 3744 ts_tablet_manager.cc:1398] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:25.753612 3742 tablet_bootstrap.cc:492] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:25.753765 3742 ts_tablet_manager.cc:1367] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:25.754033 3679 raft_consensus.cc:482] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:25.754118 3679 raft_consensus.cc:504] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:25.754650 3679 leader_election.cc:288] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:25.755404 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685e3ae151f9458f83b303061a5a4b84" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:25.755436 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685e3ae151f9458f83b303061a5a4b84" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:25.755533 672 raft_consensus.cc:2454] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:25.755797 341 leader_election.cc:302] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:25.756139 3679 raft_consensus.cc:2781] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:25.756187 3679 raft_consensus.cc:482] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:25.756119 3742 raft_consensus.cc:348] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:25.756225 3679 raft_consensus.cc:3037] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.756234 3742 raft_consensus.cc:374] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.756273 3742 raft_consensus.cc:729] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.756395 3742 consensus_queue.cc:260] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:25.756659 3742 ts_tablet_manager.cc:1398] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
W20240502 14:00:25.757299 344 leader_election.cc:341] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:00:25.757375 3679 raft_consensus.cc:504] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:25.757748 3679 leader_election.cc:288] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:25.758605 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685e3ae151f9458f83b303061a5a4b84" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:25.758724 672 raft_consensus.cc:3037] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.759466 672 raft_consensus.cc:2454] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:25.759786 3710 raft_consensus.cc:482] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:25.759902 3710 raft_consensus.cc:504] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:25.759963 341 leader_election.cc:302] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:25.759967 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685e3ae151f9458f83b303061a5a4b84" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:25.760087 3152 raft_consensus.cc:3037] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.760145 3679 raft_consensus.cc:2781] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:25.760215 3679 raft_consensus.cc:686] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:25.760360 3710 leader_election.cc:288] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:25.760350 3679 consensus_queue.cc:237] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:25.761077 3152 raft_consensus.cc:2454] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:25.762440 32733 catalog_manager.cc:5484] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:25.766144 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685e3ae151f9458f83b303061a5a4b84" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:25.766217 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685e3ae151f9458f83b303061a5a4b84" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:25.766364 672 raft_consensus.cc:2379] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2b6796cf0e2e496e9dd641beb432f485 in current term 1: Already voted for candidate f003b732dd0f46958262b42f8d20305e in this term.
I20240502 14:00:25.766673 3084 leader_election.cc:302] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485; no voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e
I20240502 14:00:25.766870 3710 raft_consensus.cc:2726] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 14:00:25.778999 3663 consensus_queue.cc:1035] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.782407 3722 consensus_queue.cc:1035] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.833616 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:25.839632 784 tablet_service.cc:1467] Processing CreateTablet for tablet 9185ea12db4b413a8c76800b828836da (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72712ccf79f7480baf5c42d1e148d845]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.839759 3262 tablet_service.cc:1467] Processing CreateTablet for tablet 9185ea12db4b413a8c76800b828836da (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72712ccf79f7480baf5c42d1e148d845]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.840006 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9185ea12db4b413a8c76800b828836da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.840086 3262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9185ea12db4b413a8c76800b828836da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.840456 386 tablet_service.cc:1467] Processing CreateTablet for tablet 544ed46acc5345f58031444b9a2ef776 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72712ccf79f7480baf5c42d1e148d845]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.840691 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 544ed46acc5345f58031444b9a2ef776. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.842471 3757 tablet_bootstrap.cc:492] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:25.842793 3744 tablet_bootstrap.cc:492] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:25.843168 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 4c27c9fb983c49a59a2f84776775e1bd (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72712ccf79f7480baf5c42d1e148d845]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.843292 3757 tablet_bootstrap.cc:654] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.843427 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c27c9fb983c49a59a2f84776775e1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.843583 3744 tablet_bootstrap.cc:654] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.839781 388 tablet_service.cc:1467] Processing CreateTablet for tablet 9185ea12db4b413a8c76800b828836da (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72712ccf79f7480baf5c42d1e148d845]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.844110 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 544ed46acc5345f58031444b9a2ef776 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72712ccf79f7480baf5c42d1e148d845]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.844280 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9185ea12db4b413a8c76800b828836da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.844331 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 544ed46acc5345f58031444b9a2ef776. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.845506 3742 tablet_bootstrap.cc:492] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:25.846478 3742 tablet_bootstrap.cc:654] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.846844 3262 tablet_service.cc:1467] Processing CreateTablet for tablet 4c27c9fb983c49a59a2f84776775e1bd (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72712ccf79f7480baf5c42d1e148d845]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.847115 3262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c27c9fb983c49a59a2f84776775e1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.847303 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 4c27c9fb983c49a59a2f84776775e1bd (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72712ccf79f7480baf5c42d1e148d845]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.847573 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c27c9fb983c49a59a2f84776775e1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.846844 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 544ed46acc5345f58031444b9a2ef776 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72712ccf79f7480baf5c42d1e148d845]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:25.848074 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 544ed46acc5345f58031444b9a2ef776. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:25.851626 3744 tablet_bootstrap.cc:492] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:25.851706 3744 ts_tablet_manager.cc:1367] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 14:00:25.852210 3761 tablet_bootstrap.cc:492] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:25.852198 3744 raft_consensus.cc:348] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.852314 3744 raft_consensus.cc:374] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.852344 3744 raft_consensus.cc:729] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.852473 3744 consensus_queue.cc:260] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.852749 3744 ts_tablet_manager.cc:1398] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:25.852914 3744 tablet_bootstrap.cc:492] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:25.855046 3761 tablet_bootstrap.cc:654] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.855260 3742 tablet_bootstrap.cc:492] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:25.855368 3742 ts_tablet_manager.cc:1367] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:25.855389 3757 tablet_bootstrap.cc:492] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:25.855482 3757 ts_tablet_manager.cc:1367] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 14:00:25.855708 3756 tablet_bootstrap.cc:492] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:25.855814 3742 raft_consensus.cc:348] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.855880 3757 raft_consensus.cc:348] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.855966 3742 raft_consensus.cc:374] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.855973 3757 raft_consensus.cc:374] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.856004 3742 raft_consensus.cc:729] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.856007 3757 raft_consensus.cc:729] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.856132 3742 consensus_queue.cc:260] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.856407 3742 ts_tablet_manager.cc:1398] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:25.856606 3742 tablet_bootstrap.cc:492] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:25.856979 3756 tablet_bootstrap.cc:654] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.857522 3742 tablet_bootstrap.cc:654] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.858096 3756 tablet_bootstrap.cc:492] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:25.858511 3756 ts_tablet_manager.cc:1367] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:25.858609 3744 tablet_bootstrap.cc:654] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.858996 3756 raft_consensus.cc:348] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.859175 3756 raft_consensus.cc:374] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.859228 3756 raft_consensus.cc:729] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.859367 3756 consensus_queue.cc:260] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.859805 3742 tablet_bootstrap.cc:492] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:25.859959 3742 ts_tablet_manager.cc:1367] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:25.859964 3744 tablet_bootstrap.cc:492] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:25.860044 3744 ts_tablet_manager.cc:1367] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:00:25.860359 3756 ts_tablet_manager.cc:1398] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:25.860440 3744 raft_consensus.cc:348] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.860517 3756 tablet_bootstrap.cc:492] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:25.860544 3744 raft_consensus.cc:374] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.860577 3744 raft_consensus.cc:729] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.860566 3742 raft_consensus.cc:348] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.860651 3742 raft_consensus.cc:374] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.860683 3742 raft_consensus.cc:729] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.860713 3744 consensus_queue.cc:260] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.860793 3742 consensus_queue.cc:260] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.861003 3744 ts_tablet_manager.cc:1398] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:25.861079 3742 ts_tablet_manager.cc:1398] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:25.861462 3756 tablet_bootstrap.cc:654] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.862814 3756 tablet_bootstrap.cc:492] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:25.862932 3756 ts_tablet_manager.cc:1367] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:25.856197 3757 consensus_queue.cc:260] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.863344 3756 raft_consensus.cc:348] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.863540 3756 raft_consensus.cc:374] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.863690 3757 ts_tablet_manager.cc:1398] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240502 14:00:25.863914 3756 raft_consensus.cc:729] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.864115 3756 consensus_queue.cc:260] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.864514 3756 ts_tablet_manager.cc:1398] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:25.864724 3756 tablet_bootstrap.cc:492] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:25.865833 3756 tablet_bootstrap.cc:654] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:25.867609 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:25.872021 672 raft_consensus.cc:1261] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:25.873512 3748 consensus_queue.cc:1035] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.873638 3761 tablet_bootstrap.cc:492] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:25.873737 3761 ts_tablet_manager.cc:1367] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240502 14:00:25.874102 3761 raft_consensus.cc:348] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.874179 3761 raft_consensus.cc:374] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.874145 3152 raft_consensus.cc:1261] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:25.874199 3761 raft_consensus.cc:729] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.874308 3761 consensus_queue.cc:260] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:25.874562 3761 ts_tablet_manager.cc:1398] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:25.875373 3756 tablet_bootstrap.cc:492] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:25.875525 3756 ts_tablet_manager.cc:1367] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:00:25.875969 3756 raft_consensus.cc:348] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.876107 3756 raft_consensus.cc:374] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:25.876171 3756 raft_consensus.cc:729] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:25.876235 3748 consensus_queue.cc:1035] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:25.876303 3756 consensus_queue.cc:260] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.876663 3756 ts_tablet_manager.cc:1398] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:25.877051 32731 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 685e3ae151f9458f83b303061a5a4b84 (table rebalancer_dynamic_table [id=9e7442eda1d343028feacdcecaeda624]). Expected version 1 got 0
I20240502 14:00:25.879448 32732 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 685e3ae151f9458f83b303061a5a4b84 (table rebalancer_dynamic_table [id=9e7442eda1d343028feacdcecaeda624]). Expected version 1 got 0
I20240502 14:00:25.880851 3729 tablet.cc:1722] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:25.881146 3729 tablet.cc:1620] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:25.881275 32732 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet 685e3ae151f9458f83b303061a5a4b84 (table rebalancer_dynamic_table [id=9e7442eda1d343028feacdcecaeda624]). Expected version 1 got 0
I20240502 14:00:25.883635 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=9e7442eda1d343028feacdcecaeda624] alter complete (version 1)
I20240502 14:00:25.884513 32731 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet 685e3ae151f9458f83b303061a5a4b84 (table rebalancer_dynamic_table [id=9e7442eda1d343028feacdcecaeda624]). Expected version 1 got 0
I20240502 14:00:25.884971 3773 tablet.cc:1722] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:25.885143 3774 tablet.cc:1722] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:25.885814 3773 tablet.cc:1620] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:25.885866 32732 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 685e3ae151f9458f83b303061a5a4b84 (table rebalancer_dynamic_table [id=9e7442eda1d343028feacdcecaeda624]). Expected version 1 got 0
I20240502 14:00:25.885922 3774 tablet.cc:1620] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:25.980229 3769 raft_consensus.cc:482] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:25.980357 3769 raft_consensus.cc:504] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.980825 3769 leader_election.cc:288] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:25.981293 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9185ea12db4b413a8c76800b828836da" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:25.981453 409 raft_consensus.cc:2454] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:25.981767 740 leader_election.cc:302] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:25.982012 3769 raft_consensus.cc:2781] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:25.982074 3769 raft_consensus.cc:482] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:25.982120 3769 raft_consensus.cc:3037] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.983186 3769 raft_consensus.cc:504] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.983520 3769 leader_election.cc:288] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:25.983882 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9185ea12db4b413a8c76800b828836da" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:25.984012 409 raft_consensus.cc:3037] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.985049 409 raft_consensus.cc:2454] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:25.985350 740 leader_election.cc:302] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:25.985468 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9185ea12db4b413a8c76800b828836da" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:25.985612 3284 raft_consensus.cc:2454] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:25.985587 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9185ea12db4b413a8c76800b828836da" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:25.985744 3283 raft_consensus.cc:3037] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:25.986521 3283 raft_consensus.cc:2454] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:25.986680 3769 raft_consensus.cc:2781] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:25.986785 3769 raft_consensus.cc:686] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:25.986925 3769 consensus_queue.cc:237] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:25.988503 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:25.988938 32731 catalog_manager.cc:5484] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:25.994086 3773 tablet.cc:1722] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:25.994529 3729 tablet.cc:1722] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:25.994858 3729 tablet.cc:1620] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:25.994951 3774 tablet.cc:1722] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:25.994500 3773 tablet.cc:1620] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:25.995225 3774 tablet.cc:1620] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:26.000156 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=9e7442eda1d343028feacdcecaeda624] alter complete (version 2)
I20240502 14:00:26.099635 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_14" } modify_external_catalogs: true
I20240502 14:00:26.099797 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_14" } modify_external_catalogs: true
I20240502 14:00:26.102279 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 73e11a7fc3cb4b96924b6d705227f438
I20240502 14:00:26.102655 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet a2dfd860f1164db198b890c2f5827051
I20240502 14:00:26.102878 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 73e11a7fc3cb4b96924b6d705227f438 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:26.102979 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 73e11a7fc3cb4b96924b6d705227f438 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:26.103025 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:26.103159 3644 tablet_replica.cc:330] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:26.103329 3644 raft_consensus.cc:2229] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.103336 386 tablet_service.cc:1514] Processing DeleteTablet for tablet a2dfd860f1164db198b890c2f5827051 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:26.103379 3780 tablet_replica.cc:330] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:26.103490 3644 raft_consensus.cc:2258] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.103500 3780 raft_consensus.cc:2229] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:26.103518 3711 tablet_replica.cc:330] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:26.103649 3711 raft_consensus.cc:2229] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.103700 3780 raft_consensus.cc:2258] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.103780 3711 raft_consensus.cc:2258] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.104228 3780 ts_tablet_manager.cc:1875] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.104321 3711 ts_tablet_manager.cc:1875] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.104898 784 tablet_service.cc:1514] Processing DeleteTablet for tablet a2dfd860f1164db198b890c2f5827051 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:26.105026 388 tablet_service.cc:1514] Processing DeleteTablet for tablet 73e11a7fc3cb4b96924b6d705227f438 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:26.105034 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet a286ed72ded04c2bb113066c7f574adf
I20240502 14:00:26.106338 784 tablet_service.cc:1514] Processing DeleteTablet for tablet a286ed72ded04c2bb113066c7f574adf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:26.104099 3644 ts_tablet_manager.cc:1875] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.106655 3711 ts_tablet_manager.cc:1888] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.106725 3711 log.cc:1196] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/a2dfd860f1164db198b890c2f5827051
I20240502 14:00:26.106745 3780 ts_tablet_manager.cc:1888] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.106824 3780 log.cc:1196] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/73e11a7fc3cb4b96924b6d705227f438
I20240502 14:00:26.107002 3711 ts_tablet_manager.cc:1909] T a2dfd860f1164db198b890c2f5827051 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:26.107147 3780 ts_tablet_manager.cc:1909] T 73e11a7fc3cb4b96924b6d705227f438 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:26.108147 388 tablet_service.cc:1514] Processing DeleteTablet for tablet a286ed72ded04c2bb113066c7f574adf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:26.108160 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet a2dfd860f1164db198b890c2f5827051 (table rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]) successfully deleted
I20240502 14:00:26.108614 3644 ts_tablet_manager.cc:1888] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.108686 3644 log.cc:1196] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/73e11a7fc3cb4b96924b6d705227f438
I20240502 14:00:26.108954 3644 ts_tablet_manager.cc:1909] T 73e11a7fc3cb4b96924b6d705227f438 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:26.109109 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 73e11a7fc3cb4b96924b6d705227f438 (table rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]) successfully deleted
I20240502 14:00:26.109114 520 tablet_service.cc:1514] Processing DeleteTablet for tablet a286ed72ded04c2bb113066c7f574adf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:26.109319 3780 tablet_replica.cc:330] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:26.109429 3780 raft_consensus.cc:2229] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:26.109614 3780 raft_consensus.cc:2258] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.109616 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 73e11a7fc3cb4b96924b6d705227f438 (table rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]) successfully deleted
I20240502 14:00:26.109922 653 tablet_service.cc:1514] Processing DeleteTablet for tablet a2dfd860f1164db198b890c2f5827051 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:26.110054 3644 tablet_replica.cc:330] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:26.110146 3780 ts_tablet_manager.cc:1875] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.110162 3644 raft_consensus.cc:2229] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.110237 3710 raft_consensus.cc:482] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:26.110322 3710 raft_consensus.cc:504] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:26.110702 3710 leader_election.cc:288] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:26.110743 3644 raft_consensus.cc:2258] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.111286 3711 tablet_replica.cc:330] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:26.111253 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "544ed46acc5345f58031444b9a2ef776" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:26.111397 3711 raft_consensus.cc:2229] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.111388 3283 raft_consensus.cc:2454] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:26.111528 3711 raft_consensus.cc:2258] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.111719 3085 leader_election.cc:302] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:26.111898 3710 raft_consensus.cc:2781] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:26.111943 3710 raft_consensus.cc:482] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:26.112031 3711 ts_tablet_manager.cc:1875] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.111964 3729 tablet.cc:1722] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:26.112098 3780 ts_tablet_manager.cc:1888] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.112154 3780 log.cc:1196] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/a286ed72ded04c2bb113066c7f574adf
I20240502 14:00:26.112453 3780 ts_tablet_manager.cc:1909] T a286ed72ded04c2bb113066c7f574adf P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:26.112965 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "544ed46acc5345f58031444b9a2ef776" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:26.113090 409 raft_consensus.cc:2454] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:26.113574 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet a286ed72ded04c2bb113066c7f574adf (table rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]) successfully deleted
I20240502 14:00:26.111987 3710 raft_consensus.cc:3037] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.114153 3711 ts_tablet_manager.cc:1888] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.114223 3711 log.cc:1196] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/73e11a7fc3cb4b96924b6d705227f438
I20240502 14:00:26.114569 3711 ts_tablet_manager.cc:1909] T 73e11a7fc3cb4b96924b6d705227f438 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:26.114708 3710 raft_consensus.cc:504] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:26.115103 3710 leader_election.cc:288] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:26.115226 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 73e11a7fc3cb4b96924b6d705227f438 (table rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]) successfully deleted
I20240502 14:00:26.115418 3711 tablet_replica.cc:330] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:26.115612 3509 raft_consensus.cc:482] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:26.115706 3509 raft_consensus.cc:504] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:26.115695 3774 tablet.cc:1722] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:26.116117 3509 leader_election.cc:288] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:26.115556 3711 raft_consensus.cc:2229] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.116329 3711 raft_consensus.cc:2258] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.116675 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c27c9fb983c49a59a2f84776775e1bd" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:26.116773 3711 ts_tablet_manager.cc:1875] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.116803 3283 raft_consensus.cc:2454] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:26.117081 2820 leader_election.cc:302] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:26.117301 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=9e7442eda1d343028feacdcecaeda624] alter complete (version 3)
I20240502 14:00:26.117756 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c27c9fb983c49a59a2f84776775e1bd" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:26.117815 3509 raft_consensus.cc:2781] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:26.117877 3509 raft_consensus.cc:482] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:26.117884 3152 raft_consensus.cc:2454] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:26.117914 3509 raft_consensus.cc:3037] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.117888 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "544ed46acc5345f58031444b9a2ef776" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:26.117986 3283 raft_consensus.cc:3037] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.118896 3283 raft_consensus.cc:2454] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:26.118914 3509 raft_consensus.cc:504] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:26.119174 3085 leader_election.cc:302] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:26.119282 3509 leader_election.cc:288] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:26.119319 3710 raft_consensus.cc:2781] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:26.119473 3710 raft_consensus.cc:686] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:26.119601 3710 consensus_queue.cc:237] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:26.119733 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c27c9fb983c49a59a2f84776775e1bd" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:26.119865 3283 raft_consensus.cc:3037] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.120059 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c27c9fb983c49a59a2f84776775e1bd" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:26.120144 3152 raft_consensus.cc:3037] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.120733 3283 raft_consensus.cc:2454] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:26.121052 3152 raft_consensus.cc:2454] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
W20240502 14:00:26.121104 608 consensus_peers.cc:485] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a2dfd860f1164db198b890c2f5827051. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:26.121510 2820 leader_election.cc:302] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:26.121675 3509 raft_consensus.cc:2781] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:26.121675 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "544ed46acc5345f58031444b9a2ef776" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:26.121766 3509 raft_consensus.cc:686] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:26.121778 409 raft_consensus.cc:3037] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.121907 3509 consensus_queue.cc:237] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:26.122661 409 raft_consensus.cc:2454] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:26.123292 32732 catalog_manager.cc:5484] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:26.123783 3711 ts_tablet_manager.cc:1888] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.123864 3711 log.cc:1196] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/a286ed72ded04c2bb113066c7f574adf
I20240502 14:00:26.124130 3711 ts_tablet_manager.cc:1909] T a286ed72ded04c2bb113066c7f574adf P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:26.124286 3781 tablet_replica.cc:330] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:26.124461 3781 raft_consensus.cc:2229] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:26.124439 32733 catalog_manager.cc:5484] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:26.124783 3781 raft_consensus.cc:2258] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.124833 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet a286ed72ded04c2bb113066c7f574adf (table rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]) successfully deleted
I20240502 14:00:26.124852 3644 ts_tablet_manager.cc:1875] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.125303 3781 ts_tablet_manager.cc:1875] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.127230 3644 ts_tablet_manager.cc:1888] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.127301 3644 log.cc:1196] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/a2dfd860f1164db198b890c2f5827051
I20240502 14:00:26.127492 3644 ts_tablet_manager.cc:1909] T a2dfd860f1164db198b890c2f5827051 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:26.127643 3773 tablet.cc:1722] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:26.128062 3781 ts_tablet_manager.cc:1888] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.128129 3781 log.cc:1196] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/a2dfd860f1164db198b890c2f5827051
I20240502 14:00:26.128445 3781 ts_tablet_manager.cc:1909] T a2dfd860f1164db198b890c2f5827051 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:26.128605 3644 tablet_replica.cc:330] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:26.128571 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet a2dfd860f1164db198b890c2f5827051 (table rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]) successfully deleted
I20240502 14:00:26.128726 3644 raft_consensus.cc:2229] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.129143 3644 raft_consensus.cc:2258] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.129151 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet a2dfd860f1164db198b890c2f5827051 (table rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]) successfully deleted
I20240502 14:00:26.129899 3644 ts_tablet_manager.cc:1875] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.131964 3644 ts_tablet_manager.cc:1888] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.132028 3644 log.cc:1196] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/a286ed72ded04c2bb113066c7f574adf
I20240502 14:00:26.132248 3644 ts_tablet_manager.cc:1909] T a286ed72ded04c2bb113066c7f574adf P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:26.132823 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet a286ed72ded04c2bb113066c7f574adf (table rebalance_test_table_14 [id=4bbe0ae584aa42d39234c11539c91b43]) successfully deleted
I20240502 14:00:26.219354 32732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:26.219506 32732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:26.221771 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 685e3ae151f9458f83b303061a5a4b84
I20240502 14:00:26.222363 388 tablet_service.cc:1514] Processing DeleteTablet for tablet 685e3ae151f9458f83b303061a5a4b84 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:26.222415 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet 685e3ae151f9458f83b303061a5a4b84 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:26.222723 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 685e3ae151f9458f83b303061a5a4b84 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:26.222910 3781 tablet_replica.cc:330] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:26.222911 3711 tablet_replica.cc:330] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:26.223053 3781 raft_consensus.cc:2229] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.223053 3711 raft_consensus.cc:2229] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:26.223177 3781 raft_consensus.cc:2258] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.223290 3711 raft_consensus.cc:2258] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.223575 3786 tablet_replica.cc:330] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:26.223935 3786 raft_consensus.cc:2229] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.224011 3781 ts_tablet_manager.cc:1875] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.224020 3711 ts_tablet_manager.cc:1875] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.224079 3786 raft_consensus.cc:2258] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.225912 3781 ts_tablet_manager.cc:1888] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:26.225912 3711 ts_tablet_manager.cc:1888] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:26.225988 3711 log.cc:1196] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/685e3ae151f9458f83b303061a5a4b84
I20240502 14:00:26.225988 3781 log.cc:1196] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/685e3ae151f9458f83b303061a5a4b84
I20240502 14:00:26.226059 3786 ts_tablet_manager.cc:1875] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.226253 3711 ts_tablet_manager.cc:1909] T 685e3ae151f9458f83b303061a5a4b84 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:26.226253 3781 ts_tablet_manager.cc:1909] T 685e3ae151f9458f83b303061a5a4b84 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:26.226992 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 685e3ae151f9458f83b303061a5a4b84 (table rebalancer_dynamic_table_new_name [id=9e7442eda1d343028feacdcecaeda624]) successfully deleted
I20240502 14:00:26.227030 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 685e3ae151f9458f83b303061a5a4b84 (table rebalancer_dynamic_table_new_name [id=9e7442eda1d343028feacdcecaeda624]) successfully deleted
I20240502 14:00:26.228137 3786 ts_tablet_manager.cc:1888] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:26.228200 3786 log.cc:1196] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/685e3ae151f9458f83b303061a5a4b84
I20240502 14:00:26.228478 3786 ts_tablet_manager.cc:1909] T 685e3ae151f9458f83b303061a5a4b84 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:26.229053 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 685e3ae151f9458f83b303061a5a4b84 (table rebalancer_dynamic_table_new_name [id=9e7442eda1d343028feacdcecaeda624]) successfully deleted
I20240502 14:00:26.323876 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:26.329208 388 tablet_service.cc:1467] Processing CreateTablet for tablet a4f97770939d41b8b78361cad06c9ac5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a302fbe9c0e94dada06181c55492ca51]), partition=
I20240502 14:00:26.329311 2999 tablet_service.cc:1467] Processing CreateTablet for tablet a4f97770939d41b8b78361cad06c9ac5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a302fbe9c0e94dada06181c55492ca51]), partition=
I20240502 14:00:26.329528 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f97770939d41b8b78361cad06c9ac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.329596 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f97770939d41b8b78361cad06c9ac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.331234 3744 tablet_bootstrap.cc:492] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:26.331341 3787 tablet_bootstrap.cc:492] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:26.331490 520 tablet_service.cc:1467] Processing CreateTablet for tablet a4f97770939d41b8b78361cad06c9ac5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a302fbe9c0e94dada06181c55492ca51]), partition=
I20240502 14:00:26.331709 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f97770939d41b8b78361cad06c9ac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.332093 3744 tablet_bootstrap.cc:654] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.332693 3787 tablet_bootstrap.cc:654] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.333175 3744 tablet_bootstrap.cc:492] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:26.333269 3744 ts_tablet_manager.cc:1367] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:26.333678 3787 tablet_bootstrap.cc:492] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:26.333741 3787 ts_tablet_manager.cc:1367] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:26.333710 3744 raft_consensus.cc:348] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:26.333818 3744 raft_consensus.cc:374] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.333863 3744 raft_consensus.cc:729] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.333985 3744 consensus_queue.cc:260] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:26.334117 3787 raft_consensus.cc:348] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:26.334225 3787 raft_consensus.cc:374] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.334231 3744 ts_tablet_manager.cc:1398] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:26.334250 3787 raft_consensus.cc:729] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.334374 3787 consensus_queue.cc:260] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:26.334631 3787 ts_tablet_manager.cc:1398] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:26.334690 3790 tablet_bootstrap.cc:492] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:26.335525 3790 tablet_bootstrap.cc:654] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.336833 3790 tablet_bootstrap.cc:492] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:26.336935 3790 ts_tablet_manager.cc:1367] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:26.337356 3790 raft_consensus.cc:348] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:26.337472 3790 raft_consensus.cc:374] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.337543 3790 raft_consensus.cc:729] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.337675 3790 consensus_queue.cc:260] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:26.337970 3790 ts_tablet_manager.cc:1398] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:26.416461 3725 raft_consensus.cc:482] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:26.416595 3725 raft_consensus.cc:504] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:26.416976 3725 leader_election.cc:288] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:26.417336 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f97770939d41b8b78361cad06c9ac5" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:26.417347 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f97770939d41b8b78361cad06c9ac5" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:26.417467 409 raft_consensus.cc:2454] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:26.417475 3019 raft_consensus.cc:2454] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:26.417752 476 leader_election.cc:302] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:26.417923 3725 raft_consensus.cc:2781] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:26.417999 3725 raft_consensus.cc:482] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:26.418030 3725 raft_consensus.cc:3037] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.418882 3725 raft_consensus.cc:504] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:26.419209 3725 leader_election.cc:288] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:26.419551 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f97770939d41b8b78361cad06c9ac5" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:26.419550 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f97770939d41b8b78361cad06c9ac5" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:26.419667 3019 raft_consensus.cc:3037] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.419667 409 raft_consensus.cc:3037] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.420619 3019 raft_consensus.cc:2454] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:26.420619 409 raft_consensus.cc:2454] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:26.420905 476 leader_election.cc:302] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:26.421063 3725 raft_consensus.cc:2781] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:26.421164 3725 raft_consensus.cc:686] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:26.421303 3725 consensus_queue.cc:237] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:26.423092 32732 catalog_manager.cc:5484] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:26.450157 3769 consensus_queue.cc:1035] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:26.453970 3684 consensus_queue.cc:1035] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:26.505085 3509 consensus_queue.cc:1035] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:26.508363 3503 consensus_queue.cc:1035] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:26.528985 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:26.533995 3019 raft_consensus.cc:1261] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:26.533995 409 raft_consensus.cc:1261] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:26.534431 3725 consensus_queue.cc:1035] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:26.534794 3785 consensus_queue.cc:1035] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:26.537114 32732 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet a4f97770939d41b8b78361cad06c9ac5 (table rebalancer_dynamic_table [id=a302fbe9c0e94dada06181c55492ca51]). Expected version 1 got 0
I20240502 14:00:26.537516 3804 tablet.cc:1722] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:26.537853 3804 tablet.cc:1620] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:26.538259 32733 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet a4f97770939d41b8b78361cad06c9ac5 (table rebalancer_dynamic_table [id=a302fbe9c0e94dada06181c55492ca51]). Expected version 1 got 0
I20240502 14:00:26.538800 3805 tablet.cc:1722] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:26.539036 3773 tablet.cc:1722] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:26.539140 3805 tablet.cc:1620] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:26.539326 3773 tablet.cc:1620] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:26.540184 3710 consensus_queue.cc:1035] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:26.543635 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=a302fbe9c0e94dada06181c55492ca51] alter complete (version 1)
I20240502 14:00:26.545518 3782 consensus_queue.cc:1035] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:26.606650 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_16" } modify_external_catalogs: true
I20240502 14:00:26.606817 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_16" } modify_external_catalogs: true
I20240502 14:00:26.609182 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 68992bd623d848f390b36e093890fa70
I20240502 14:00:26.609524 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet edad7a65b4114f348c4b5a5754213120
I20240502 14:00:26.609797 388 tablet_service.cc:1514] Processing DeleteTablet for tablet 68992bd623d848f390b36e093890fa70 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:26.609813 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 68992bd623d848f390b36e093890fa70 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:26.610000 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 68992bd623d848f390b36e093890fa70 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:26.610021 3780 tablet_replica.cc:330] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:26.610076 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 12dc7df342664dbb9a50921736baf935
I20240502 14:00:26.610172 3781 tablet_replica.cc:330] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:26.610184 3780 raft_consensus.cc:2229] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.610298 3781 raft_consensus.cc:2229] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:26.610344 3780 raft_consensus.cc:2258] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.610509 3781 raft_consensus.cc:2258] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.610787 388 tablet_service.cc:1514] Processing DeleteTablet for tablet edad7a65b4114f348c4b5a5754213120 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:26.610895 3780 ts_tablet_manager.cc:1875] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.611054 3781 ts_tablet_manager.cc:1875] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.611353 784 tablet_service.cc:1514] Processing DeleteTablet for tablet edad7a65b4114f348c4b5a5754213120 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:26.611383 786 tablet_service.cc:1514] Processing DeleteTablet for tablet 12dc7df342664dbb9a50921736baf935 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:26.611527 3644 tablet_replica.cc:330] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:26.611647 3644 raft_consensus.cc:2229] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:26.611856 3644 raft_consensus.cc:2258] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.612108 386 tablet_service.cc:1514] Processing DeleteTablet for tablet 12dc7df342664dbb9a50921736baf935 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:26.612285 3711 tablet_replica.cc:330] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:26.612385 3644 ts_tablet_manager.cc:1875] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.612414 3711 raft_consensus.cc:2229] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.612551 3711 raft_consensus.cc:2258] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.613065 3711 ts_tablet_manager.cc:1875] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.613271 3780 ts_tablet_manager.cc:1888] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.613337 3780 log.cc:1196] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/68992bd623d848f390b36e093890fa70
I20240502 14:00:26.613384 3781 ts_tablet_manager.cc:1888] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.613447 3781 log.cc:1196] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/68992bd623d848f390b36e093890fa70
I20240502 14:00:26.613626 3780 ts_tablet_manager.cc:1909] T 68992bd623d848f390b36e093890fa70 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:26.613726 3781 ts_tablet_manager.cc:1909] T 68992bd623d848f390b36e093890fa70 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:26.614655 3644 ts_tablet_manager.cc:1888] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.614706 3644 log.cc:1196] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/edad7a65b4114f348c4b5a5754213120
I20240502 14:00:26.614926 3644 ts_tablet_manager.cc:1909] T edad7a65b4114f348c4b5a5754213120 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:26.615077 3711 ts_tablet_manager.cc:1888] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.615126 3711 log.cc:1196] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/68992bd623d848f390b36e093890fa70
I20240502 14:00:26.615366 3711 ts_tablet_manager.cc:1909] T 68992bd623d848f390b36e093890fa70 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:26.615809 3644 tablet_replica.cc:330] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:26.615978 3644 raft_consensus.cc:2229] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.616050 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet edad7a65b4114f348c4b5a5754213120 (table rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]) successfully deleted
I20240502 14:00:26.616189 3644 raft_consensus.cc:2258] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.616261 520 tablet_service.cc:1514] Processing DeleteTablet for tablet edad7a65b4114f348c4b5a5754213120 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:26.616473 3780 tablet_replica.cc:330] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:26.616592 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 68992bd623d848f390b36e093890fa70 (table rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]) successfully deleted
I20240502 14:00:26.616633 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 12dc7df342664dbb9a50921736baf935 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:26.616735 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 68992bd623d848f390b36e093890fa70 (table rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]) successfully deleted
I20240502 14:00:26.616784 3781 tablet_replica.cc:330] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:26.616823 3711 tablet_replica.cc:330] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:26.616868 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 68992bd623d848f390b36e093890fa70 (table rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]) successfully deleted
I20240502 14:00:26.616899 3780 raft_consensus.cc:2229] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.616945 3781 raft_consensus.cc:2229] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:26.616937 3711 raft_consensus.cc:2229] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.617039 3780 raft_consensus.cc:2258] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.617059 3711 raft_consensus.cc:2258] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.617112 3781 raft_consensus.cc:2258] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.617398 3780 ts_tablet_manager.cc:1875] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.617518 3781 ts_tablet_manager.cc:1875] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.617511 3711 ts_tablet_manager.cc:1875] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.618903 3780 ts_tablet_manager.cc:1888] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.618954 3780 log.cc:1196] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/edad7a65b4114f348c4b5a5754213120
I20240502 14:00:26.619166 3780 ts_tablet_manager.cc:1909] T edad7a65b4114f348c4b5a5754213120 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:26.619297 3711 ts_tablet_manager.cc:1888] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.619297 3781 ts_tablet_manager.cc:1888] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.619360 3711 log.cc:1196] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/edad7a65b4114f348c4b5a5754213120
I20240502 14:00:26.619360 3781 log.cc:1196] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/12dc7df342664dbb9a50921736baf935
I20240502 14:00:26.619621 3781 ts_tablet_manager.cc:1909] T 12dc7df342664dbb9a50921736baf935 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:26.619621 3711 ts_tablet_manager.cc:1909] T edad7a65b4114f348c4b5a5754213120 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:26.619796 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet edad7a65b4114f348c4b5a5754213120 (table rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]) successfully deleted
I20240502 14:00:26.619916 3644 ts_tablet_manager.cc:1875] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.620379 3711 tablet_replica.cc:330] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:26.620455 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet edad7a65b4114f348c4b5a5754213120 (table rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]) successfully deleted
I20240502 14:00:26.620525 3711 raft_consensus.cc:2229] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.620693 3711 raft_consensus.cc:2258] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.621189 3711 ts_tablet_manager.cc:1875] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.621919 3644 ts_tablet_manager.cc:1888] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.622006 3644 log.cc:1196] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/12dc7df342664dbb9a50921736baf935
I20240502 14:00:26.622291 3644 ts_tablet_manager.cc:1909] T 12dc7df342664dbb9a50921736baf935 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:26.622923 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 12dc7df342664dbb9a50921736baf935 (table rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]) successfully deleted
I20240502 14:00:26.623046 3711 ts_tablet_manager.cc:1888] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:26.623121 3711 log.cc:1196] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/12dc7df342664dbb9a50921736baf935
I20240502 14:00:26.623361 3711 ts_tablet_manager.cc:1909] T 12dc7df342664dbb9a50921736baf935 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:26.623349 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 12dc7df342664dbb9a50921736baf935 (table rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]) successfully deleted
I20240502 14:00:26.623956 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 12dc7df342664dbb9a50921736baf935 (table rebalance_test_table_16 [id=81030ab151fb45fc8329a127ff449788]) successfully deleted
I20240502 14:00:26.638391 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:26.643918 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 527784cdca2f4840aa2f2a373a2c6db6 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=3efb8e76f34447f4adfad33ea46ac121]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:26.644258 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 527784cdca2f4840aa2f2a373a2c6db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.644670 520 tablet_service.cc:1467] Processing CreateTablet for tablet f78ea0e379fa4f9786c7b2e45c9c8cc9 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=3efb8e76f34447f4adfad33ea46ac121]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:26.644915 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f78ea0e379fa4f9786c7b2e45c9c8cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.646345 3808 tablet_bootstrap.cc:492] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:26.646551 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 527784cdca2f4840aa2f2a373a2c6db6 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=3efb8e76f34447f4adfad33ea46ac121]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:26.646754 3790 tablet_bootstrap.cc:492] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:26.646826 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 527784cdca2f4840aa2f2a373a2c6db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.647372 3808 tablet_bootstrap.cc:654] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.647464 3790 tablet_bootstrap.cc:654] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.648679 3790 tablet_bootstrap.cc:492] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:26.648694 3808 tablet_bootstrap.cc:492] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:26.648739 3790 ts_tablet_manager.cc:1367] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:26.648774 3808 ts_tablet_manager.cc:1367] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:26.649142 3811 tablet_bootstrap.cc:492] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:26.649129 3790 raft_consensus.cc:348] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.649245 3790 raft_consensus.cc:374] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.649276 3790 raft_consensus.cc:729] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.649389 3790 consensus_queue.cc:260] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.649550 3808 raft_consensus.cc:348] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:26.649682 3790 ts_tablet_manager.cc:1398] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:26.649703 3808 raft_consensus.cc:374] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.649739 3808 raft_consensus.cc:729] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.649874 3808 consensus_queue.cc:260] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:26.649900 386 tablet_service.cc:1467] Processing CreateTablet for tablet 527784cdca2f4840aa2f2a373a2c6db6 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=3efb8e76f34447f4adfad33ea46ac121]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:26.650158 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 527784cdca2f4840aa2f2a373a2c6db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.650158 3808 ts_tablet_manager.cc:1398] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:26.650171 3811 tablet_bootstrap.cc:654] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.652315 3744 tablet_bootstrap.cc:492] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:26.653057 3744 tablet_bootstrap.cc:654] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.653496 3811 tablet_bootstrap.cc:492] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:26.654151 3744 tablet_bootstrap.cc:492] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:26.654235 3744 ts_tablet_manager.cc:1367] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:26.654664 3744 raft_consensus.cc:348] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:26.654760 3744 raft_consensus.cc:374] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.654814 3744 raft_consensus.cc:729] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.654918 3744 consensus_queue.cc:260] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:26.655186 3744 ts_tablet_manager.cc:1398] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:26.656386 786 tablet_service.cc:1467] Processing CreateTablet for tablet bbae9f2a88db4a92b89cc12dfb837ad2 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=3efb8e76f34447f4adfad33ea46ac121]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:26.656425 2866 tablet_service.cc:1467] Processing CreateTablet for tablet bbae9f2a88db4a92b89cc12dfb837ad2 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=3efb8e76f34447f4adfad33ea46ac121]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:26.656675 786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae9f2a88db4a92b89cc12dfb837ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.656683 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae9f2a88db4a92b89cc12dfb837ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.658402 3814 tablet_bootstrap.cc:492] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:26.658557 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:26.656428 784 tablet_service.cc:1467] Processing CreateTablet for tablet f78ea0e379fa4f9786c7b2e45c9c8cc9 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=3efb8e76f34447f4adfad33ea46ac121]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:26.659119 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f78ea0e379fa4f9786c7b2e45c9c8cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.659332 3814 tablet_bootstrap.cc:654] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.660003 2864 tablet_service.cc:1467] Processing CreateTablet for tablet f78ea0e379fa4f9786c7b2e45c9c8cc9 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=3efb8e76f34447f4adfad33ea46ac121]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:26.660233 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f78ea0e379fa4f9786c7b2e45c9c8cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.660930 3130 tablet_service.cc:1467] Processing CreateTablet for tablet bbae9f2a88db4a92b89cc12dfb837ad2 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=3efb8e76f34447f4adfad33ea46ac121]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:26.661124 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae9f2a88db4a92b89cc12dfb837ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.663264 3808 tablet_bootstrap.cc:492] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:26.664216 3808 tablet_bootstrap.cc:654] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.666152 3804 tablet.cc:1722] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:26.666411 3804 tablet.cc:1620] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:26.666735 3808 tablet_bootstrap.cc:492] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:26.666822 3808 ts_tablet_manager.cc:1367] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:26.667135 3773 tablet.cc:1722] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:26.667295 3808 raft_consensus.cc:348] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.667400 3808 raft_consensus.cc:374] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.667407 3773 tablet.cc:1620] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:26.667438 3808 raft_consensus.cc:729] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.667570 3808 consensus_queue.cc:260] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.668062 3808 ts_tablet_manager.cc:1398] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:26.668926 3815 tablet_bootstrap.cc:492] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:26.671876 3811 ts_tablet_manager.cc:1367] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240502 14:00:26.672482 3811 raft_consensus.cc:348] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:26.672683 3811 raft_consensus.cc:374] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.672765 3811 raft_consensus.cc:729] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.672808 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=a302fbe9c0e94dada06181c55492ca51] alter complete (version 2)
I20240502 14:00:26.673012 3811 consensus_queue.cc:260] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:26.673621 3815 tablet_bootstrap.cc:654] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.673854 3811 ts_tablet_manager.cc:1398] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:26.675449 3805 tablet.cc:1722] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:26.675769 3805 tablet.cc:1620] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:26.676138 3815 tablet_bootstrap.cc:492] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:26.676229 3815 ts_tablet_manager.cc:1367] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:26.676718 3815 raft_consensus.cc:348] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.676820 3815 raft_consensus.cc:374] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.676860 3815 raft_consensus.cc:729] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.676992 3815 consensus_queue.cc:260] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.677256 3815 ts_tablet_manager.cc:1398] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:26.677379 3815 tablet_bootstrap.cc:492] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:26.678211 3815 tablet_bootstrap.cc:654] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.680305 3815 tablet_bootstrap.cc:492] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:26.680389 3815 ts_tablet_manager.cc:1367] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:26.681146 3815 raft_consensus.cc:348] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.681248 3815 raft_consensus.cc:374] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.681285 3815 raft_consensus.cc:729] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.681448 3815 consensus_queue.cc:260] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.681787 3815 ts_tablet_manager.cc:1398] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:26.682932 3814 tablet_bootstrap.cc:492] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:26.683091 3814 ts_tablet_manager.cc:1367] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240502 14:00:26.683595 3814 raft_consensus.cc:348] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.683903 3814 raft_consensus.cc:374] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.683971 3814 raft_consensus.cc:729] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.684111 3814 consensus_queue.cc:260] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.684419 3814 ts_tablet_manager.cc:1398] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:26.684657 3814 tablet_bootstrap.cc:492] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:26.685947 3814 tablet_bootstrap.cc:654] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:26.687111 3814 tablet_bootstrap.cc:492] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:26.687249 3814 ts_tablet_manager.cc:1367] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:26.687659 3814 raft_consensus.cc:348] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.687745 3814 raft_consensus.cc:374] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:26.687780 3814 raft_consensus.cc:729] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:26.687942 3814 consensus_queue.cc:260] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.688271 3814 ts_tablet_manager.cc:1398] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:26.712428 3782 raft_consensus.cc:482] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:26.712553 3782 raft_consensus.cc:504] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:26.712951 3782 leader_election.cc:288] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:26.713295 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "527784cdca2f4840aa2f2a373a2c6db6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:26.713451 409 raft_consensus.cc:2454] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:26.713435 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "527784cdca2f4840aa2f2a373a2c6db6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:26.713610 3283 raft_consensus.cc:2454] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:26.713776 3086 leader_election.cc:302] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:26.713995 3782 raft_consensus.cc:2781] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:26.714058 3782 raft_consensus.cc:482] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:26.714112 3782 raft_consensus.cc:3037] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.715140 3782 raft_consensus.cc:504] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:26.715478 3782 leader_election.cc:288] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:26.715754 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "527784cdca2f4840aa2f2a373a2c6db6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:26.715770 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "527784cdca2f4840aa2f2a373a2c6db6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:26.715898 409 raft_consensus.cc:3037] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.715904 3283 raft_consensus.cc:3037] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.716791 409 raft_consensus.cc:2454] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:26.716781 3283 raft_consensus.cc:2454] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:26.717089 3086 leader_election.cc:302] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:26.717240 3782 raft_consensus.cc:2781] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:26.717348 3782 raft_consensus.cc:686] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:26.717490 3782 consensus_queue.cc:237] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:26.719262 32733 catalog_manager.cc:5484] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:26.753741 3769 raft_consensus.cc:482] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:26.753863 3769 raft_consensus.cc:504] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.754364 3769 leader_election.cc:288] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:26.754755 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78ea0e379fa4f9786c7b2e45c9c8cc9" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:26.754753 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78ea0e379fa4f9786c7b2e45c9c8cc9" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:26.754899 2886 raft_consensus.cc:2454] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:26.754909 541 raft_consensus.cc:2454] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:26.755234 741 leader_election.cc:302] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:26.755403 3769 raft_consensus.cc:2781] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:26.755458 3769 raft_consensus.cc:482] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:26.755498 3769 raft_consensus.cc:3037] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.756467 3769 raft_consensus.cc:504] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.756804 3769 leader_election.cc:288] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:26.757151 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78ea0e379fa4f9786c7b2e45c9c8cc9" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:26.757261 541 raft_consensus.cc:3037] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.758029 541 raft_consensus.cc:2454] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:26.758296 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78ea0e379fa4f9786c7b2e45c9c8cc9" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:26.758335 738 leader_election.cc:302] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:26.758414 2886 raft_consensus.cc:3037] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.758502 3769 raft_consensus.cc:2781] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:26.758580 3769 raft_consensus.cc:686] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:26.758736 3769 consensus_queue.cc:237] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.759398 2886 raft_consensus.cc:2454] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:26.760679 32733 catalog_manager.cc:5484] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:26.767282 3710 raft_consensus.cc:482] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:26.767390 3710 raft_consensus.cc:504] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.767808 3710 leader_election.cc:288] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:26.768177 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae9f2a88db4a92b89cc12dfb837ad2" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:26.768322 2886 raft_consensus.cc:2454] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:26.768674 3087 leader_election.cc:302] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:26.768867 3710 raft_consensus.cc:2781] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:26.768932 3710 raft_consensus.cc:482] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:26.769001 3710 raft_consensus.cc:3037] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.769829 3710 raft_consensus.cc:504] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.770267 3710 leader_election.cc:288] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:26.770970 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae9f2a88db4a92b89cc12dfb837ad2" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:26.771072 2886 raft_consensus.cc:3037] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.771983 2886 raft_consensus.cc:2454] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:26.772118 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae9f2a88db4a92b89cc12dfb837ad2" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:26.772089 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbae9f2a88db4a92b89cc12dfb837ad2" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:26.772220 804 raft_consensus.cc:3037] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:26.772233 3087 leader_election.cc:302] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:26.772387 3710 raft_consensus.cc:2781] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:26.772471 3710 raft_consensus.cc:686] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:26.772593 3710 consensus_queue.cc:237] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:26.773133 804 raft_consensus.cc:2454] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:26.774245 32733 catalog_manager.cc:5484] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:26.775838 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:26.780894 3804 tablet.cc:1722] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:26.781565 3773 tablet.cc:1722] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:26.781669 3805 tablet.cc:1722] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:26.785836 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=a302fbe9c0e94dada06181c55492ca51] alter complete (version 3)
I20240502 14:00:26.888013 32732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:26.888199 32732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:26.890223 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet a4f97770939d41b8b78361cad06c9ac5
I20240502 14:00:26.890807 386 tablet_service.cc:1514] Processing DeleteTablet for tablet a4f97770939d41b8b78361cad06c9ac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:26.890928 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet a4f97770939d41b8b78361cad06c9ac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:26.890928 520 tablet_service.cc:1514] Processing DeleteTablet for tablet a4f97770939d41b8b78361cad06c9ac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:26 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:26.891038 3711 tablet_replica.cc:330] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:26.891181 3711 raft_consensus.cc:2229] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.891192 3780 tablet_replica.cc:330] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:26.891243 3827 tablet_replica.cc:330] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:26.891311 3711 raft_consensus.cc:2258] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.891335 3780 raft_consensus.cc:2229] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:26.891386 3827 raft_consensus.cc:2229] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:26.891541 3780 raft_consensus.cc:2258] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.891543 3827 raft_consensus.cc:2258] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:26.892004 3711 ts_tablet_manager.cc:1875] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.892068 3780 ts_tablet_manager.cc:1875] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.892192 3827 ts_tablet_manager.cc:1875] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:26.893824 3780 ts_tablet_manager.cc:1888] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:26.893893 3780 log.cc:1196] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/a4f97770939d41b8b78361cad06c9ac5
I20240502 14:00:26.894129 3780 ts_tablet_manager.cc:1909] T a4f97770939d41b8b78361cad06c9ac5 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:26.894258 3711 ts_tablet_manager.cc:1888] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:26.894337 3711 log.cc:1196] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/a4f97770939d41b8b78361cad06c9ac5
I20240502 14:00:26.894379 3827 ts_tablet_manager.cc:1888] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:26.894456 3827 log.cc:1196] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/a4f97770939d41b8b78361cad06c9ac5
I20240502 14:00:26.894631 3711 ts_tablet_manager.cc:1909] T a4f97770939d41b8b78361cad06c9ac5 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:26.894906 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet a4f97770939d41b8b78361cad06c9ac5 (table rebalancer_dynamic_table_new_name [id=a302fbe9c0e94dada06181c55492ca51]) successfully deleted
I20240502 14:00:26.894981 3827 ts_tablet_manager.cc:1909] T a4f97770939d41b8b78361cad06c9ac5 P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:26.895350 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet a4f97770939d41b8b78361cad06c9ac5 (table rebalancer_dynamic_table_new_name [id=a302fbe9c0e94dada06181c55492ca51]) successfully deleted
I20240502 14:00:26.895641 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet a4f97770939d41b8b78361cad06c9ac5 (table rebalancer_dynamic_table_new_name [id=a302fbe9c0e94dada06181c55492ca51]) successfully deleted
I20240502 14:00:26.992033 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:26.997686 3130 tablet_service.cc:1467] Processing CreateTablet for tablet ebc01f2ae1b143449e5ff812a50fe0b6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c64671a4733140708e11b597816d1945]), partition=
I20240502 14:00:26.998013 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc01f2ae1b143449e5ff812a50fe0b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:26.998047 2999 tablet_service.cc:1467] Processing CreateTablet for tablet ebc01f2ae1b143449e5ff812a50fe0b6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c64671a4733140708e11b597816d1945]), partition=
I20240502 14:00:26.998282 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc01f2ae1b143449e5ff812a50fe0b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.000283 3808 tablet_bootstrap.cc:492] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:27.000519 386 tablet_service.cc:1467] Processing CreateTablet for tablet ebc01f2ae1b143449e5ff812a50fe0b6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c64671a4733140708e11b597816d1945]), partition=
I20240502 14:00:27.000545 3829 tablet_bootstrap.cc:492] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:27.000751 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc01f2ae1b143449e5ff812a50fe0b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.001264 3808 tablet_bootstrap.cc:654] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.001547 3829 tablet_bootstrap.cc:654] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.002497 3744 tablet_bootstrap.cc:492] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:27.002759 3829 tablet_bootstrap.cc:492] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:27.002822 3829 ts_tablet_manager.cc:1367] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:27.003161 3829 raft_consensus.cc:348] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.003270 3829 raft_consensus.cc:374] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.003295 3829 raft_consensus.cc:729] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.003329 3744 tablet_bootstrap.cc:654] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.003392 3829 consensus_queue.cc:260] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.003662 3829 ts_tablet_manager.cc:1398] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:27.004491 3744 tablet_bootstrap.cc:492] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:27.004607 3744 ts_tablet_manager.cc:1367] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:27.005040 3744 raft_consensus.cc:348] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.005157 3744 raft_consensus.cc:374] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.005198 3744 raft_consensus.cc:729] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.005316 3744 consensus_queue.cc:260] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.005594 3744 ts_tablet_manager.cc:1398] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:27.020176 3808 tablet_bootstrap.cc:492] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:27.020272 3808 ts_tablet_manager.cc:1367] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 14:00:27.020697 3808 raft_consensus.cc:348] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.020797 3808 raft_consensus.cc:374] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.020828 3808 raft_consensus.cc:729] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.020953 3808 consensus_queue.cc:260] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.021209 3808 ts_tablet_manager.cc:1398] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:27.116398 32732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_18" } modify_external_catalogs: true
I20240502 14:00:27.116544 32732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalance_test_table_18" } modify_external_catalogs: true
I20240502 14:00:27.118885 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 8e38b601db0c4a51aa08f1f94107331e
I20240502 14:00:27.119423 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e38b601db0c4a51aa08f1f94107331e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:27.119506 386 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e38b601db0c4a51aa08f1f94107331e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:27.119491 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e38b601db0c4a51aa08f1f94107331e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:27.119624 3780 tablet_replica.cc:330] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:27.119685 3644 tablet_replica.cc:330] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:27.119699 3711 tablet_replica.cc:330] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:27.119772 3780 raft_consensus.cc:2229] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:27.119838 3644 raft_consensus.cc:2229] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:27.119855 3711 raft_consensus.cc:2229] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:27.119915 3780 raft_consensus.cc:2258] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.119978 3711 raft_consensus.cc:2258] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.120066 3644 raft_consensus.cc:2258] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.120460 3780 ts_tablet_manager.cc:1875] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.120512 3711 ts_tablet_manager.cc:1875] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.120501 3644 ts_tablet_manager.cc:1875] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.122455 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 1db6563013284583925202aeedc631ed
I20240502 14:00:27.122618 3780 ts_tablet_manager.cc:1888] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:27.122685 3780 log.cc:1196] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/8e38b601db0c4a51aa08f1f94107331e
I20240502 14:00:27.122668 3711 ts_tablet_manager.cc:1888] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:27.122673 3644 ts_tablet_manager.cc:1888] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:27.122735 3711 log.cc:1196] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/8e38b601db0c4a51aa08f1f94107331e
I20240502 14:00:27.122733 3644 log.cc:1196] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/8e38b601db0c4a51aa08f1f94107331e
I20240502 14:00:27.123004 3780 ts_tablet_manager.cc:1909] T 8e38b601db0c4a51aa08f1f94107331e P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:27.122984 386 tablet_service.cc:1514] Processing DeleteTablet for tablet 1db6563013284583925202aeedc631ed with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:27.123023 3644 ts_tablet_manager.cc:1909] T 8e38b601db0c4a51aa08f1f94107331e P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:27.123028 3711 ts_tablet_manager.cc:1909] T 8e38b601db0c4a51aa08f1f94107331e P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:27.123791 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 8e38b601db0c4a51aa08f1f94107331e (table rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]) successfully deleted
I20240502 14:00:27.123993 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 1db6563013284583925202aeedc631ed with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:27.124171 3644 tablet_replica.cc:330] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:27.124363 3644 raft_consensus.cc:2229] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:27.124627 3644 raft_consensus.cc:2258] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.124943 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 1db6563013284583925202aeedc631ed with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:27.125103 3711 tablet_replica.cc:330] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:27.125190 3644 ts_tablet_manager.cc:1875] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.125224 3711 raft_consensus.cc:2229] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:27.125214 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 8e38b601db0c4a51aa08f1f94107331e (table rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]) successfully deleted
I20240502 14:00:27.125391 3711 raft_consensus.cc:2258] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.125855 3711 ts_tablet_manager.cc:1875] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.126394 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 8e38b601db0c4a51aa08f1f94107331e (table rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]) successfully deleted
I20240502 14:00:27.127171 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet d10d62c409d542ed8f80b952ccc48826
I20240502 14:00:27.127315 3644 ts_tablet_manager.cc:1888] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:27.127436 3644 log.cc:1196] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/1db6563013284583925202aeedc631ed
I20240502 14:00:27.127714 520 tablet_service.cc:1514] Processing DeleteTablet for tablet d10d62c409d542ed8f80b952ccc48826 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:27.127866 386 tablet_service.cc:1514] Processing DeleteTablet for tablet d10d62c409d542ed8f80b952ccc48826 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:27.127902 3780 tablet_replica.cc:330] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:27.127936 3711 ts_tablet_manager.cc:1888] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:27.128029 3780 raft_consensus.cc:2229] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:27.128063 3834 tablet_replica.cc:330] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:27.128203 3780 raft_consensus.cc:2258] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.128221 3834 raft_consensus.cc:2229] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:27.128291 784 tablet_service.cc:1514] Processing DeleteTablet for tablet d10d62c409d542ed8f80b952ccc48826 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:27.128371 3834 raft_consensus.cc:2258] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.128011 3711 log.cc:1196] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/1db6563013284583925202aeedc631ed
I20240502 14:00:27.128690 3711 ts_tablet_manager.cc:1909] T 1db6563013284583925202aeedc631ed P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:27.128808 3834 ts_tablet_manager.cc:1875] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.128808 3780 ts_tablet_manager.cc:1875] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.129195 3711 tablet_replica.cc:330] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:27.129312 3711 raft_consensus.cc:2229] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:27.129303 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 1db6563013284583925202aeedc631ed (table rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]) successfully deleted
I20240502 14:00:27.129479 3711 raft_consensus.cc:2258] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.129868 3711 ts_tablet_manager.cc:1875] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.130949 3834 ts_tablet_manager.cc:1888] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:27.131064 3834 log.cc:1196] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/1db6563013284583925202aeedc631ed
I20240502 14:00:27.131094 3780 ts_tablet_manager.cc:1888] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:27.131179 3780 log.cc:1196] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/d10d62c409d542ed8f80b952ccc48826
I20240502 14:00:27.131376 3834 ts_tablet_manager.cc:1909] T 1db6563013284583925202aeedc631ed P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:27.131470 3780 ts_tablet_manager.cc:1909] T d10d62c409d542ed8f80b952ccc48826 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:27.131625 3711 ts_tablet_manager.cc:1888] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:27.131688 3711 log.cc:1196] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/d10d62c409d542ed8f80b952ccc48826
I20240502 14:00:27.127777 3644 ts_tablet_manager.cc:1909] T 1db6563013284583925202aeedc631ed P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:27.131968 3711 ts_tablet_manager.cc:1909] T d10d62c409d542ed8f80b952ccc48826 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:27.132196 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet d10d62c409d542ed8f80b952ccc48826 (table rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]) successfully deleted
I20240502 14:00:27.132341 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 1db6563013284583925202aeedc631ed (table rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]) successfully deleted
I20240502 14:00:27.132597 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet d10d62c409d542ed8f80b952ccc48826 (table rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]) successfully deleted
I20240502 14:00:27.132639 3644 tablet_replica.cc:330] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:27.132735 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 1db6563013284583925202aeedc631ed (table rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]) successfully deleted
I20240502 14:00:27.132757 3644 raft_consensus.cc:2229] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:27.132906 3644 raft_consensus.cc:2258] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.133284 3644 ts_tablet_manager.cc:1875] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.134716 3644 ts_tablet_manager.cc:1888] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 14:00:27.134804 3644 log.cc:1196] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/d10d62c409d542ed8f80b952ccc48826
I20240502 14:00:27.135032 3644 ts_tablet_manager.cc:1909] T d10d62c409d542ed8f80b952ccc48826 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:27.135593 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet d10d62c409d542ed8f80b952ccc48826 (table rebalance_test_table_18 [id=77184b3f2c6945df9734273ec030484a]) successfully deleted
I20240502 14:00:27.169641 3826 consensus_queue.cc:1035] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.173096 3769 consensus_queue.cc:1035] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.229689 3828 consensus_queue.cc:1035] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.232667 3828 consensus_queue.cc:1035] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.238481 3782 consensus_queue.cc:1035] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.248029 3828 consensus_queue.cc:1035] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.258702 3782 raft_consensus.cc:482] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:27.258821 3782 raft_consensus.cc:504] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.259260 3782 leader_election.cc:288] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:27.259692 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc01f2ae1b143449e5ff812a50fe0b6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:27.259860 405 raft_consensus.cc:2454] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:27.260164 3086 leader_election.cc:302] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:27.260341 3782 raft_consensus.cc:2781] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:27.260411 3782 raft_consensus.cc:482] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:27.260473 3782 raft_consensus.cc:3037] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.261319 3782 raft_consensus.cc:504] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.261646 3782 leader_election.cc:288] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:27.261991 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc01f2ae1b143449e5ff812a50fe0b6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:27.262109 405 raft_consensus.cc:3037] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.263022 405 raft_consensus.cc:2454] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:27.263224 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc01f2ae1b143449e5ff812a50fe0b6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:27.263300 3086 leader_election.cc:302] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:27.263303 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc01f2ae1b143449e5ff812a50fe0b6" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:27.263368 3019 raft_consensus.cc:2454] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:27.263458 3782 raft_consensus.cc:2781] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:27.263551 3782 raft_consensus.cc:686] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:27.263667 3782 consensus_queue.cc:237] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.265336 32732 catalog_manager.cc:5484] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:27.295985 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:27.301414 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 638cb0f08414491ea80139eb51406dfe (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=4e2ce13a9a4c4a03a79ac1f8bfba20bc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:27.301498 386 tablet_service.cc:1467] Processing CreateTablet for tablet 638cb0f08414491ea80139eb51406dfe (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=4e2ce13a9a4c4a03a79ac1f8bfba20bc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:27.301764 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638cb0f08414491ea80139eb51406dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.301798 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638cb0f08414491ea80139eb51406dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.302148 2866 tablet_service.cc:1467] Processing CreateTablet for tablet a4015371eba240cb8e8f7e2180ac3d27 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=4e2ce13a9a4c4a03a79ac1f8bfba20bc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:27.302389 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4015371eba240cb8e8f7e2180ac3d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.302346 2864 tablet_service.cc:1467] Processing CreateTablet for tablet 8849d10396af4bf8b5ff047f93a631b1 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=4e2ce13a9a4c4a03a79ac1f8bfba20bc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:27.302587 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8849d10396af4bf8b5ff047f93a631b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.302889 784 tablet_service.cc:1467] Processing CreateTablet for tablet 638cb0f08414491ea80139eb51406dfe (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=4e2ce13a9a4c4a03a79ac1f8bfba20bc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:27.303153 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638cb0f08414491ea80139eb51406dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.303977 3829 tablet_bootstrap.cc:492] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:27.304042 3744 tablet_bootstrap.cc:492] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:27.304899 3744 tablet_bootstrap.cc:654] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.304960 3829 tablet_bootstrap.cc:654] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.305555 3847 tablet_bootstrap.cc:492] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:27.305560 386 tablet_service.cc:1467] Processing CreateTablet for tablet a4015371eba240cb8e8f7e2180ac3d27 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=4e2ce13a9a4c4a03a79ac1f8bfba20bc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:27.305761 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4015371eba240cb8e8f7e2180ac3d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.305881 3848 tablet_bootstrap.cc:492] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:27.305904 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 8849d10396af4bf8b5ff047f93a631b1 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=4e2ce13a9a4c4a03a79ac1f8bfba20bc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:27.306161 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8849d10396af4bf8b5ff047f93a631b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.306515 3847 tablet_bootstrap.cc:654] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.306690 3848 tablet_bootstrap.cc:654] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.307648 3847 tablet_bootstrap.cc:492] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:27.307642 3132 tablet_service.cc:1467] Processing CreateTablet for tablet a4015371eba240cb8e8f7e2180ac3d27 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=4e2ce13a9a4c4a03a79ac1f8bfba20bc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:27.307723 3847 ts_tablet_manager.cc:1367] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:27.307866 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4015371eba240cb8e8f7e2180ac3d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.308214 3847 raft_consensus.cc:348] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:27.308326 3847 raft_consensus.cc:374] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.308363 3847 raft_consensus.cc:729] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.308506 3847 consensus_queue.cc:260] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:27.308816 3847 ts_tablet_manager.cc:1398] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:27.309244 3808 tablet_bootstrap.cc:492] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:27.309561 3829 tablet_bootstrap.cc:492] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:27.309638 3829 ts_tablet_manager.cc:1367] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:27.310091 3829 raft_consensus.cc:348] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:27.310209 3808 tablet_bootstrap.cc:654] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.310211 3829 raft_consensus.cc:374] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.310243 3829 raft_consensus.cc:729] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.310375 3829 consensus_queue.cc:260] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:27.310654 3829 ts_tablet_manager.cc:1398] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:27.310990 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 8849d10396af4bf8b5ff047f93a631b1 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=4e2ce13a9a4c4a03a79ac1f8bfba20bc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:27.311259 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8849d10396af4bf8b5ff047f93a631b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.312073 3808 tablet_bootstrap.cc:492] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:27.312157 3808 ts_tablet_manager.cc:1367] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:27.312331 3847 tablet_bootstrap.cc:492] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:27.312578 3744 tablet_bootstrap.cc:492] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:27.312589 3808 raft_consensus.cc:348] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:27.312681 3808 raft_consensus.cc:374] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.312685 3744 ts_tablet_manager.cc:1367] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 14:00:27.312717 3808 raft_consensus.cc:729] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.312841 3808 consensus_queue.cc:260] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:27.313086 3808 ts_tablet_manager.cc:1398] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:27.313169 3744 raft_consensus.cc:348] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:27.313241 3847 tablet_bootstrap.cc:654] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.313269 3808 tablet_bootstrap.cc:492] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:27.313273 3744 raft_consensus.cc:374] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.313311 3744 raft_consensus.cc:729] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.313441 3744 consensus_queue.cc:260] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:27.313736 3744 ts_tablet_manager.cc:1398] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:27.313943 3744 tablet_bootstrap.cc:492] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:27.314174 3808 tablet_bootstrap.cc:654] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.314603 3848 tablet_bootstrap.cc:492] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:27.314699 3848 ts_tablet_manager.cc:1367] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:27.314898 3744 tablet_bootstrap.cc:654] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.315186 3848 raft_consensus.cc:348] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:27.315306 3848 raft_consensus.cc:374] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.315351 3848 raft_consensus.cc:729] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.315778 3848 consensus_queue.cc:260] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:27.316269 3847 tablet_bootstrap.cc:492] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:27.316340 3847 ts_tablet_manager.cc:1367] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:27.316787 3847 raft_consensus.cc:348] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:27.316890 3847 raft_consensus.cc:374] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.316939 3847 raft_consensus.cc:729] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.317046 3847 consensus_queue.cc:260] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:27.317325 3847 ts_tablet_manager.cc:1398] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:27.317629 3848 ts_tablet_manager.cc:1398] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:27.318327 3808 tablet_bootstrap.cc:492] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:27.318404 3808 ts_tablet_manager.cc:1367] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:27.318395 3744 tablet_bootstrap.cc:492] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:27.318487 3744 ts_tablet_manager.cc:1367] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:27.318816 3808 raft_consensus.cc:348] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:27.318920 3808 raft_consensus.cc:374] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.318953 3808 raft_consensus.cc:729] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.318902 3744 raft_consensus.cc:348] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:27.318998 3744 raft_consensus.cc:374] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.319032 3744 raft_consensus.cc:729] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.319072 3808 consensus_queue.cc:260] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:27.319160 3744 consensus_queue.cc:260] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:27.319316 3808 ts_tablet_manager.cc:1398] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:27.319437 3744 ts_tablet_manager.cc:1398] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:27.325420 3857 tablet_bootstrap.cc:492] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:27.326334 3857 tablet_bootstrap.cc:654] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.327431 3857 tablet_bootstrap.cc:492] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:27.327524 3857 ts_tablet_manager.cc:1367] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:27.328013 3857 raft_consensus.cc:348] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:27.328141 3857 raft_consensus.cc:374] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.328174 3857 raft_consensus.cc:729] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.328296 3857 consensus_queue.cc:260] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:27.328548 3857 ts_tablet_manager.cc:1398] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:27.348678 3732 raft_consensus.cc:482] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:27.348802 3732 raft_consensus.cc:504] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:27.349268 3732 leader_election.cc:288] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:27.349689 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638cb0f08414491ea80139eb51406dfe" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:27.349848 806 raft_consensus.cc:2454] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:27.350127 2954 leader_election.cc:302] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:27.350299 3732 raft_consensus.cc:2781] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:27.350348 3732 raft_consensus.cc:482] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:27.350379 3732 raft_consensus.cc:3037] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.351428 3732 raft_consensus.cc:504] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:27.351768 3732 leader_election.cc:288] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:27.352157 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638cb0f08414491ea80139eb51406dfe" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:27.352272 806 raft_consensus.cc:3037] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.353235 806 raft_consensus.cc:2454] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:27.353540 2954 leader_election.cc:302] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:27.353701 3732 raft_consensus.cc:2781] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:27.353875 3732 raft_consensus.cc:686] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:27.354022 3732 consensus_queue.cc:237] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:27.354014 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638cb0f08414491ea80139eb51406dfe" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:27.354096 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638cb0f08414491ea80139eb51406dfe" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:27.354153 405 raft_consensus.cc:2454] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:27.355726 32732 catalog_manager.cc:5484] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:27.395941 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:27.399783 409 raft_consensus.cc:1261] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:27.400002 3018 raft_consensus.cc:3037] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.400221 3828 consensus_queue.cc:1035] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.402478 32732 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet ebc01f2ae1b143449e5ff812a50fe0b6 (table rebalancer_dynamic_table [id=c64671a4733140708e11b597816d1945]). Expected version 1 got 0
I20240502 14:00:27.402937 3865 tablet.cc:1722] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:27.403442 3865 tablet.cc:1620] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:27.403638 32731 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet ebc01f2ae1b143449e5ff812a50fe0b6 (table rebalancer_dynamic_table [id=c64671a4733140708e11b597816d1945]). Expected version 1 got 0
I20240502 14:00:27.404843 32731 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet ebc01f2ae1b143449e5ff812a50fe0b6 (table rebalancer_dynamic_table [id=c64671a4733140708e11b597816d1945]). Expected version 1 got 0
I20240502 14:00:27.405323 3866 tablet.cc:1722] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:27.405633 3866 tablet.cc:1620] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:27.408180 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=c64671a4733140708e11b597816d1945] alter complete (version 1)
I20240502 14:00:27.420732 3018 raft_consensus.cc:1261] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:27.421191 3782 consensus_queue.cc:1035] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.422719 3869 tablet.cc:1722] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:27.423097 3869 tablet.cc:1620] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:27.423918 32730 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet ebc01f2ae1b143449e5ff812a50fe0b6 (table rebalancer_dynamic_table [id=c64671a4733140708e11b597816d1945]). Expected version 1 got 0
I20240502 14:00:27.511322 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:27.516753 32730 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet ebc01f2ae1b143449e5ff812a50fe0b6 (table rebalancer_dynamic_table [id=c64671a4733140708e11b597816d1945]). Expected version 2 got 1
I20240502 14:00:27.516736 3865 tablet.cc:1722] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:27.517102 3865 tablet.cc:1620] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:27.517920 3869 tablet.cc:1722] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:27.518098 3866 tablet.cc:1722] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:27.518214 3869 tablet.cc:1620] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:27.518368 3866 tablet.cc:1620] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:27.522715 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=c64671a4733140708e11b597816d1945] alter complete (version 2)
I20240502 14:00:27.530243 3690 raft_consensus.cc:482] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:27.530344 3690 raft_consensus.cc:504] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:27.530771 3690 leader_election.cc:288] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:27.531129 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8849d10396af4bf8b5ff047f93a631b1" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:27.531155 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8849d10396af4bf8b5ff047f93a631b1" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:27.531256 3152 raft_consensus.cc:2454] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:27.531288 2886 raft_consensus.cc:2454] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:27.531543 3219 leader_election.cc:302] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:27.532014 3690 raft_consensus.cc:2781] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:27.532085 3690 raft_consensus.cc:482] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:27.532138 3690 raft_consensus.cc:3037] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.533104 3690 raft_consensus.cc:504] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:27.533402 3690 leader_election.cc:288] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:27.533769 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8849d10396af4bf8b5ff047f93a631b1" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:27.533823 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8849d10396af4bf8b5ff047f93a631b1" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:27.533875 2886 raft_consensus.cc:3037] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.533943 3152 raft_consensus.cc:3037] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.534611 2886 raft_consensus.cc:2454] T 8849d10396af4bf8b5ff047f93a631b1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:27.534659 3152 raft_consensus.cc:2454] T 8849d10396af4bf8b5ff047f93a631b1 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:27.534904 3219 leader_election.cc:302] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:27.535058 3690 raft_consensus.cc:2781] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:27.535218 3690 raft_consensus.cc:686] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:27.535347 3690 consensus_queue.cc:237] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:27.536998 32730 catalog_manager.cc:5484] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:27.556267 3710 raft_consensus.cc:482] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:27.556370 3710 raft_consensus.cc:504] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:27.556783 3710 leader_election.cc:288] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:27.557180 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4015371eba240cb8e8f7e2180ac3d27" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:27.557226 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4015371eba240cb8e8f7e2180ac3d27" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:27.557304 2886 raft_consensus.cc:2454] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:27.557379 409 raft_consensus.cc:2454] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:27.557621 3087 leader_election.cc:302] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:27.557811 3710 raft_consensus.cc:2781] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:27.557878 3710 raft_consensus.cc:482] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:27.557905 3710 raft_consensus.cc:3037] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.558779 3710 raft_consensus.cc:504] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:27.559124 3710 leader_election.cc:288] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:27.559410 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4015371eba240cb8e8f7e2180ac3d27" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:27.559537 2886 raft_consensus.cc:3037] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.559562 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4015371eba240cb8e8f7e2180ac3d27" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:27.559662 409 raft_consensus.cc:3037] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.560562 2886 raft_consensus.cc:2454] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:27.560593 409 raft_consensus.cc:2454] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:27.560846 3086 leader_election.cc:302] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:27.561039 3710 raft_consensus.cc:2781] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:27.561120 3710 raft_consensus.cc:686] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:27.561260 3710 consensus_queue.cc:237] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:27.562990 32730 catalog_manager.cc:5484] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:27.624578 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:27.629902 3865 tablet.cc:1722] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:27.630533 3869 tablet.cc:1722] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:27.630546 3866 tablet.cc:1722] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:27.635685 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=c64671a4733140708e11b597816d1945] alter complete (version 3)
I20240502 14:00:27.737738 32733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:27.737898 32733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:27.739269 3732 consensus_queue.cc:1035] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.740154 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet ebc01f2ae1b143449e5ff812a50fe0b6
I20240502 14:00:27.740746 386 tablet_service.cc:1514] Processing DeleteTablet for tablet ebc01f2ae1b143449e5ff812a50fe0b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:27.740752 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet ebc01f2ae1b143449e5ff812a50fe0b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:27.741165 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet ebc01f2ae1b143449e5ff812a50fe0b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:27 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:27.741240 3875 tablet_replica.cc:330] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:27.741375 3875 raft_consensus.cc:2229] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:27.741564 3874 tablet_replica.cc:330] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:27.741586 3875 raft_consensus.cc:2258] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.741976 3874 raft_consensus.cc:2229] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:27.742189 3874 raft_consensus.cc:2258] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.742228 3875 ts_tablet_manager.cc:1875] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.743075 409 raft_consensus.cc:3037] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.743338 3876 tablet_replica.cc:330] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:27.743525 3876 raft_consensus.cc:2229] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:27.744798 3862 consensus_queue.cc:1035] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.745018 3876 raft_consensus.cc:2258] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:27.745402 3874 ts_tablet_manager.cc:1875] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.745690 3875 ts_tablet_manager.cc:1888] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:27.745821 3875 log.cc:1196] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/ebc01f2ae1b143449e5ff812a50fe0b6
I20240502 14:00:27.746131 3875 ts_tablet_manager.cc:1909] T ebc01f2ae1b143449e5ff812a50fe0b6 P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:27.747426 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet ebc01f2ae1b143449e5ff812a50fe0b6 (table rebalancer_dynamic_table_new_name [id=c64671a4733140708e11b597816d1945]) successfully deleted
I20240502 14:00:27.748931 3874 ts_tablet_manager.cc:1888] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:27.749019 3874 log.cc:1196] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/ebc01f2ae1b143449e5ff812a50fe0b6
I20240502 14:00:27.749074 3876 ts_tablet_manager.cc:1875] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:27.749369 3874 ts_tablet_manager.cc:1909] T ebc01f2ae1b143449e5ff812a50fe0b6 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:27.749991 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet ebc01f2ae1b143449e5ff812a50fe0b6 (table rebalancer_dynamic_table_new_name [id=c64671a4733140708e11b597816d1945]) successfully deleted
I20240502 14:00:27.750983 3876 ts_tablet_manager.cc:1888] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:27.751046 3876 log.cc:1196] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/ebc01f2ae1b143449e5ff812a50fe0b6
I20240502 14:00:27.751315 3876 ts_tablet_manager.cc:1909] T ebc01f2ae1b143449e5ff812a50fe0b6 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:27.751986 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet ebc01f2ae1b143449e5ff812a50fe0b6 (table rebalancer_dynamic_table_new_name [id=c64671a4733140708e11b597816d1945]) successfully deleted
I20240502 14:00:27.842105 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:27.847039 2999 tablet_service.cc:1467] Processing CreateTablet for tablet f4778267dfb94bb09f55658cfb1cab05 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=61a183543bfb400898b3dc51e09ce540]), partition=
I20240502 14:00:27.847100 3263 tablet_service.cc:1467] Processing CreateTablet for tablet f4778267dfb94bb09f55658cfb1cab05 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=61a183543bfb400898b3dc51e09ce540]), partition=
I20240502 14:00:27.847270 3132 tablet_service.cc:1467] Processing CreateTablet for tablet f4778267dfb94bb09f55658cfb1cab05 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=61a183543bfb400898b3dc51e09ce540]), partition=
I20240502 14:00:27.847337 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4778267dfb94bb09f55658cfb1cab05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.847337 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4778267dfb94bb09f55658cfb1cab05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.847499 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4778267dfb94bb09f55658cfb1cab05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:27.849745 3880 tablet_bootstrap.cc:492] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:27.849944 3882 tablet_bootstrap.cc:492] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:27.850960 3882 tablet_bootstrap.cc:654] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.851095 3880 tablet_bootstrap.cc:654] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.851207 3881 tablet_bootstrap.cc:492] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:27.852015 3881 tablet_bootstrap.cc:654] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:27.852325 3880 tablet_bootstrap.cc:492] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:27.852463 3880 ts_tablet_manager.cc:1367] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:27.852947 3880 raft_consensus.cc:348] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.853091 3880 raft_consensus.cc:374] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.853137 3880 raft_consensus.cc:729] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.853320 3880 consensus_queue.cc:260] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.853644 3881 tablet_bootstrap.cc:492] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:27.853753 3881 ts_tablet_manager.cc:1367] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:27.853799 3880 ts_tablet_manager.cc:1398] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:27.854321 3882 tablet_bootstrap.cc:492] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:27.854468 3882 ts_tablet_manager.cc:1367] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:27.854565 3881 raft_consensus.cc:348] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.854703 3881 raft_consensus.cc:374] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.854766 3881 raft_consensus.cc:729] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.854988 3882 raft_consensus.cc:348] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.855021 3881 consensus_queue.cc:260] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.855124 3882 raft_consensus.cc:374] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:27.855177 3882 raft_consensus.cc:729] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:27.855304 3881 ts_tablet_manager.cc:1398] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:27.855482 3882 consensus_queue.cc:260] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.855599 3690 raft_consensus.cc:482] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:27.855665 3690 raft_consensus.cc:504] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.856069 3690 leader_election.cc:288] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:27.856436 3882 ts_tablet_manager.cc:1398] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:27.856963 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4778267dfb94bb09f55658cfb1cab05" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:27.856998 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4778267dfb94bb09f55658cfb1cab05" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:27.857086 3152 raft_consensus.cc:2454] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:27.857084 3018 raft_consensus.cc:2454] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:27.857373 3216 leader_election.cc:302] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:27.857558 3690 raft_consensus.cc:2781] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:27.857610 3690 raft_consensus.cc:482] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:27.857654 3690 raft_consensus.cc:3037] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.858594 3690 raft_consensus.cc:504] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.858945 3690 leader_election.cc:288] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:27.859402 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4778267dfb94bb09f55658cfb1cab05" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:27.859443 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4778267dfb94bb09f55658cfb1cab05" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:27.859498 3152 raft_consensus.cc:3037] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.859536 3018 raft_consensus.cc:3037] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:27.860440 3018 raft_consensus.cc:2454] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:27.860462 3152 raft_consensus.cc:2454] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:27.860703 3216 leader_election.cc:302] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:27.861035 3690 raft_consensus.cc:2781] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:27.861141 3690 raft_consensus.cc:686] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:27.861267 3690 consensus_queue.cc:237] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:27.862823 32733 catalog_manager.cc:5484] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:27.953600 3690 consensus_queue.cc:1035] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.957293 3870 consensus_queue.cc:1035] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.968540 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:27.972113 3152 raft_consensus.cc:1261] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:27.972201 3018 raft_consensus.cc:1261] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:27.972496 3889 consensus_queue.cc:1035] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.972625 3870 consensus_queue.cc:1035] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:27.974843 32731 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet f4778267dfb94bb09f55658cfb1cab05 (table rebalancer_dynamic_table [id=61a183543bfb400898b3dc51e09ce540]). Expected version 1 got 0
I20240502 14:00:27.975203 32733 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet f4778267dfb94bb09f55658cfb1cab05 (table rebalancer_dynamic_table [id=61a183543bfb400898b3dc51e09ce540]). Expected version 1 got 0
I20240502 14:00:27.976094 3865 tablet.cc:1722] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:27.976426 3865 tablet.cc:1620] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:27.977186 32733 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet f4778267dfb94bb09f55658cfb1cab05 (table rebalancer_dynamic_table [id=61a183543bfb400898b3dc51e09ce540]). Expected version 1 got 0
I20240502 14:00:27.978031 3869 tablet.cc:1722] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:27.978302 3869 tablet.cc:1620] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:27.979974 32733 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet f4778267dfb94bb09f55658cfb1cab05 (table rebalancer_dynamic_table [id=61a183543bfb400898b3dc51e09ce540]). Expected version 1 got 0
I20240502 14:00:27.980459 3894 tablet.cc:1722] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:27.980752 3894 tablet.cc:1620] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:27.981889 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=61a183543bfb400898b3dc51e09ce540] alter complete (version 1)
I20240502 14:00:28.055012 3828 consensus_queue.cc:1035] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.058790 3782 consensus_queue.cc:1035] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.085484 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:28.091922 3894 tablet.cc:1722] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:28.092228 3894 tablet.cc:1620] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:28.092588 3869 tablet.cc:1722] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:28.092854 3869 tablet.cc:1620] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:28.092864 3865 tablet.cc:1722] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:28.093173 3865 tablet.cc:1620] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:28.097036 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=61a183543bfb400898b3dc51e09ce540] alter complete (version 2)
I20240502 14:00:28.107280 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:28.113716 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 844d8895be4b4635bc21650b60049409 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=ad01ae0cb7ff4e6394c729fa2c5c8e27]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.114076 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 844d8895be4b4635bc21650b60049409. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.114251 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 20937116074f456ea732396f80bd7d6a (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=ad01ae0cb7ff4e6394c729fa2c5c8e27]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.114248 386 tablet_service.cc:1467] Processing CreateTablet for tablet 20937116074f456ea732396f80bd7d6a (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=ad01ae0cb7ff4e6394c729fa2c5c8e27]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.114410 388 tablet_service.cc:1467] Processing CreateTablet for tablet 844d8895be4b4635bc21650b60049409 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=ad01ae0cb7ff4e6394c729fa2c5c8e27]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.114519 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20937116074f456ea732396f80bd7d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.114542 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20937116074f456ea732396f80bd7d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.114651 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 844d8895be4b4635bc21650b60049409. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.116479 3881 tablet_bootstrap.cc:492] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:28.116869 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 844d8895be4b4635bc21650b60049409 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=ad01ae0cb7ff4e6394c729fa2c5c8e27]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.117007 3129 tablet_service.cc:1467] Processing CreateTablet for tablet 2423568c1fef4f118afa96ad22747159 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=ad01ae0cb7ff4e6394c729fa2c5c8e27]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.117110 3899 tablet_bootstrap.cc:492] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:28.117144 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 844d8895be4b4635bc21650b60049409. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.117252 3129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2423568c1fef4f118afa96ad22747159. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.117403 3880 tablet_bootstrap.cc:492] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:28.117477 3881 tablet_bootstrap.cc:654] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.117995 3899 tablet_bootstrap.cc:654] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.118387 3880 tablet_bootstrap.cc:654] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.118536 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 2423568c1fef4f118afa96ad22747159 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=ad01ae0cb7ff4e6394c729fa2c5c8e27]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.118796 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2423568c1fef4f118afa96ad22747159. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.120011 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 20937116074f456ea732396f80bd7d6a (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=ad01ae0cb7ff4e6394c729fa2c5c8e27]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.120055 3262 tablet_service.cc:1467] Processing CreateTablet for tablet 2423568c1fef4f118afa96ad22747159 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=ad01ae0cb7ff4e6394c729fa2c5c8e27]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.120345 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20937116074f456ea732396f80bd7d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.120548 3262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2423568c1fef4f118afa96ad22747159. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.121292 3880 tablet_bootstrap.cc:492] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:28.121366 3880 ts_tablet_manager.cc:1367] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:28.121825 3880 raft_consensus.cc:348] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.121923 3880 raft_consensus.cc:374] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.121963 3880 raft_consensus.cc:729] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.122078 3880 consensus_queue.cc:260] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.122376 3880 ts_tablet_manager.cc:1398] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.122558 3899 tablet_bootstrap.cc:492] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:28.122593 3882 tablet_bootstrap.cc:492] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:28.122640 3899 ts_tablet_manager.cc:1367] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:28.123131 3899 raft_consensus.cc:348] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.123283 3899 raft_consensus.cc:374] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.123338 3899 raft_consensus.cc:729] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.123526 3882 tablet_bootstrap.cc:654] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.123495 3899 consensus_queue.cc:260] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.123845 3899 ts_tablet_manager.cc:1398] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:28.124293 3881 tablet_bootstrap.cc:492] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:28.124390 3881 ts_tablet_manager.cc:1367] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:28.124594 3899 tablet_bootstrap.cc:492] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:28.124840 3881 raft_consensus.cc:348] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.124919 3881 raft_consensus.cc:374] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.124939 3881 raft_consensus.cc:729] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.125047 3881 consensus_queue.cc:260] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.125308 3881 ts_tablet_manager.cc:1398] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:28.125512 3899 tablet_bootstrap.cc:654] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.125619 3903 tablet_bootstrap.cc:492] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:28.126448 3882 tablet_bootstrap.cc:492] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:28.126538 3882 ts_tablet_manager.cc:1367] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:28.126737 3903 tablet_bootstrap.cc:654] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.127004 3882 raft_consensus.cc:348] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.127128 3882 raft_consensus.cc:374] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.127163 3882 raft_consensus.cc:729] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.127290 3882 consensus_queue.cc:260] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.127554 3882 ts_tablet_manager.cc:1398] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.127735 3882 tablet_bootstrap.cc:492] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:28.129038 3882 tablet_bootstrap.cc:654] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.131071 3880 tablet_bootstrap.cc:492] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:28.131966 3880 tablet_bootstrap.cc:654] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.133241 3882 tablet_bootstrap.cc:492] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:28.133337 3882 ts_tablet_manager.cc:1367] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:28.133680 3882 raft_consensus.cc:348] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.133766 3882 raft_consensus.cc:374] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.133810 3882 raft_consensus.cc:729] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.133930 3882 consensus_queue.cc:260] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.134191 3882 ts_tablet_manager.cc:1398] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:28.134980 3880 tablet_bootstrap.cc:492] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:28.135087 3880 ts_tablet_manager.cc:1367] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:28.135152 3899 tablet_bootstrap.cc:492] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:28.135234 3899 ts_tablet_manager.cc:1367] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 14:00:28.135499 3880 raft_consensus.cc:348] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.135593 3880 raft_consensus.cc:374] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.135627 3880 raft_consensus.cc:729] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.135654 3899 raft_consensus.cc:348] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.135743 3899 raft_consensus.cc:374] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.135778 3899 raft_consensus.cc:729] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.135753 3880 consensus_queue.cc:260] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.135928 3899 consensus_queue.cc:260] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.136090 3880 ts_tablet_manager.cc:1398] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:28.136248 3880 tablet_bootstrap.cc:492] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:28.136319 3899 ts_tablet_manager.cc:1398] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.137133 3880 tablet_bootstrap.cc:654] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.138337 3880 tablet_bootstrap.cc:492] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:28.138443 3880 ts_tablet_manager.cc:1367] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:28.138916 3880 raft_consensus.cc:348] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.139037 3880 raft_consensus.cc:374] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.139079 3880 raft_consensus.cc:729] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.139212 3880 consensus_queue.cc:260] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.139472 3880 ts_tablet_manager.cc:1398] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:28.144192 3903 tablet_bootstrap.cc:492] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:28.144284 3903 ts_tablet_manager.cc:1367] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 14:00:28.144757 3903 raft_consensus.cc:348] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.144873 3903 raft_consensus.cc:374] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.144899 3903 raft_consensus.cc:729] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.145007 3903 consensus_queue.cc:260] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.145269 3903 ts_tablet_manager.cc:1398] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:28.198525 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:28.203703 3894 tablet.cc:1722] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:28.204285 3869 tablet.cc:1722] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:28.204711 3865 tablet.cc:1722] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:28.208909 3870 raft_consensus.cc:482] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:28.209010 3870 raft_consensus.cc:504] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.209434 3870 leader_election.cc:288] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:28.209756 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=61a183543bfb400898b3dc51e09ce540] alter complete (version 3)
I20240502 14:00:28.209775 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2423568c1fef4f118afa96ad22747159" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:28.209836 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2423568c1fef4f118afa96ad22747159" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:28.209929 2886 raft_consensus.cc:2454] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:28.209978 3152 raft_consensus.cc:2454] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:28.210448 3219 leader_election.cc:302] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:28.210631 3870 raft_consensus.cc:2781] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:28.210687 3870 raft_consensus.cc:482] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:28.210716 3870 raft_consensus.cc:3037] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.211548 3870 raft_consensus.cc:504] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.211879 3870 leader_election.cc:288] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:28.212191 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2423568c1fef4f118afa96ad22747159" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:28.212325 2886 raft_consensus.cc:3037] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.212301 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2423568c1fef4f118afa96ad22747159" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:28.212410 3152 raft_consensus.cc:3037] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.213119 2886 raft_consensus.cc:2454] T 2423568c1fef4f118afa96ad22747159 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:28.213359 3152 raft_consensus.cc:2454] T 2423568c1fef4f118afa96ad22747159 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:28.213419 3219 leader_election.cc:302] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:28.213588 3870 raft_consensus.cc:2781] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:28.213678 3870 raft_consensus.cc:686] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:28.213819 3870 consensus_queue.cc:237] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.215590 32733 catalog_manager.cc:5484] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:28.283550 3782 raft_consensus.cc:482] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:28.283669 3782 raft_consensus.cc:504] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.284099 3782 leader_election.cc:288] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:28.284508 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20937116074f456ea732396f80bd7d6a" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:28.284498 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20937116074f456ea732396f80bd7d6a" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:28.284647 409 raft_consensus.cc:2454] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:28.284647 3283 raft_consensus.cc:2454] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:28.284971 3085 leader_election.cc:302] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:28.285163 3710 raft_consensus.cc:2781] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:28.285224 3710 raft_consensus.cc:482] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:28.285256 3710 raft_consensus.cc:3037] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.286125 3710 raft_consensus.cc:504] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.286494 3710 leader_election.cc:288] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:28.286798 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20937116074f456ea732396f80bd7d6a" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:28.286820 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20937116074f456ea732396f80bd7d6a" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:28.286906 3283 raft_consensus.cc:3037] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.286952 409 raft_consensus.cc:3037] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.287662 3283 raft_consensus.cc:2454] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:28.287765 409 raft_consensus.cc:2454] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:28.288020 3085 leader_election.cc:302] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:28.288190 3710 raft_consensus.cc:2781] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:28.288273 3710 raft_consensus.cc:686] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:28.288412 3710 consensus_queue.cc:237] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.290208 32733 catalog_manager.cc:5484] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:28.304186 3862 raft_consensus.cc:482] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:28.304301 3862 raft_consensus.cc:504] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.304680 3862 leader_election.cc:288] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:28.305053 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "844d8895be4b4635bc21650b60049409" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:28.305083 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "844d8895be4b4635bc21650b60049409" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:28.305192 3152 raft_consensus.cc:2454] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:28.305192 409 raft_consensus.cc:2454] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:28.305512 2953 leader_election.cc:302] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:28.305673 3862 raft_consensus.cc:2781] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:28.305733 3862 raft_consensus.cc:482] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:28.305786 3862 raft_consensus.cc:3037] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.306623 3862 raft_consensus.cc:504] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.306941 3862 leader_election.cc:288] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:28.307194 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "844d8895be4b4635bc21650b60049409" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:28.307289 3151 raft_consensus.cc:3037] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.307296 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "844d8895be4b4635bc21650b60049409" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:28.307420 409 raft_consensus.cc:3037] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.308169 3151 raft_consensus.cc:2454] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:28.308228 409 raft_consensus.cc:2454] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:28.308490 2954 leader_election.cc:302] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:28.308658 3862 raft_consensus.cc:2781] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:28.308755 3862 raft_consensus.cc:686] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:28.308869 3862 consensus_queue.cc:237] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:28.310494 32733 catalog_manager.cc:5484] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:28.311760 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:28.311897 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:28.314513 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet f4778267dfb94bb09f55658cfb1cab05
I20240502 14:00:28.315127 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet f4778267dfb94bb09f55658cfb1cab05 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:28 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:28.315774 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet f4778267dfb94bb09f55658cfb1cab05 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:28 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:28.315879 3915 tablet_replica.cc:330] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:28.316026 3915 raft_consensus.cc:2229] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:28.316211 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet f4778267dfb94bb09f55658cfb1cab05 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:28 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:28.316460 3915 raft_consensus.cc:2258] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:28.316478 3916 tablet_replica.cc:330] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:28.316504 3917 tablet_replica.cc:330] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:28.316615 3916 raft_consensus.cc:2229] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:28.316627 3917 raft_consensus.cc:2229] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:28.316737 3917 raft_consensus.cc:2258] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:28.317416 3915 ts_tablet_manager.cc:1875] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:28.317613 3916 raft_consensus.cc:2258] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:28.317731 3917 ts_tablet_manager.cc:1875] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:28.318223 3916 ts_tablet_manager.cc:1875] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:28.319891 3917 ts_tablet_manager.cc:1888] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:28.319892 3915 ts_tablet_manager.cc:1888] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:28.319972 3917 log.cc:1196] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/f4778267dfb94bb09f55658cfb1cab05
I20240502 14:00:28.319972 3915 log.cc:1196] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/f4778267dfb94bb09f55658cfb1cab05
I20240502 14:00:28.320061 3916 ts_tablet_manager.cc:1888] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:28.320145 3916 log.cc:1196] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/f4778267dfb94bb09f55658cfb1cab05
I20240502 14:00:28.320269 3915 ts_tablet_manager.cc:1909] T f4778267dfb94bb09f55658cfb1cab05 P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:28.320331 3917 ts_tablet_manager.cc:1909] T f4778267dfb94bb09f55658cfb1cab05 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:28.320438 3916 ts_tablet_manager.cc:1909] T f4778267dfb94bb09f55658cfb1cab05 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:28.320936 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet f4778267dfb94bb09f55658cfb1cab05 (table rebalancer_dynamic_table_new_name [id=61a183543bfb400898b3dc51e09ce540]) successfully deleted
I20240502 14:00:28.321038 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet f4778267dfb94bb09f55658cfb1cab05 (table rebalancer_dynamic_table_new_name [id=61a183543bfb400898b3dc51e09ce540]) successfully deleted
I20240502 14:00:28.321175 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet f4778267dfb94bb09f55658cfb1cab05 (table rebalancer_dynamic_table_new_name [id=61a183543bfb400898b3dc51e09ce540]) successfully deleted
I20240502 14:00:28.417291 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:28.422235 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 513f0378456840fcbb6a89832334f532 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1a46c1e427b948dcb5ca91483ae436f5]), partition=
I20240502 14:00:28.422550 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513f0378456840fcbb6a89832334f532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.422530 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 513f0378456840fcbb6a89832334f532 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1a46c1e427b948dcb5ca91483ae436f5]), partition=
I20240502 14:00:28.422760 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513f0378456840fcbb6a89832334f532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.424325 3882 tablet_bootstrap.cc:492] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:28.424719 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 513f0378456840fcbb6a89832334f532 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1a46c1e427b948dcb5ca91483ae436f5]), partition=
I20240502 14:00:28.424954 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513f0378456840fcbb6a89832334f532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.425287 3882 tablet_bootstrap.cc:654] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.426383 3882 tablet_bootstrap.cc:492] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:28.426468 3882 ts_tablet_manager.cc:1367] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:28.426934 3882 raft_consensus.cc:348] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.427043 3882 raft_consensus.cc:374] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.427080 3882 raft_consensus.cc:729] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.427093 3880 tablet_bootstrap.cc:492] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:28.427186 3903 tablet_bootstrap.cc:492] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:28.427199 3882 consensus_queue.cc:260] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.427469 3882 ts_tablet_manager.cc:1398] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:28.427951 3880 tablet_bootstrap.cc:654] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.428100 3903 tablet_bootstrap.cc:654] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.429379 3880 tablet_bootstrap.cc:492] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:28.429387 3903 tablet_bootstrap.cc:492] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:28.429466 3903 ts_tablet_manager.cc:1367] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:28.429466 3880 ts_tablet_manager.cc:1367] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:28.429917 3880 raft_consensus.cc:348] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.429912 3903 raft_consensus.cc:348] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.430022 3880 raft_consensus.cc:374] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.430022 3903 raft_consensus.cc:374] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.430055 3903 raft_consensus.cc:729] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.430055 3880 raft_consensus.cc:729] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.430176 3903 consensus_queue.cc:260] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.430176 3880 consensus_queue.cc:260] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.430442 3903 ts_tablet_manager.cc:1398] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:28.430442 3880 ts_tablet_manager.cc:1398] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:28.567749 3896 raft_consensus.cc:482] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:28.567898 3896 raft_consensus.cc:504] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.568369 3896 leader_election.cc:288] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:28.568727 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513f0378456840fcbb6a89832334f532" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:28.568848 2886 raft_consensus.cc:2454] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:28.568914 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513f0378456840fcbb6a89832334f532" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:28.569065 3151 raft_consensus.cc:2454] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:28.569178 3219 leader_election.cc:302] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:28.569347 3896 raft_consensus.cc:2781] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:28.569413 3896 raft_consensus.cc:482] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:28.569440 3896 raft_consensus.cc:3037] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.570298 3896 raft_consensus.cc:504] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.570607 3896 leader_election.cc:288] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:28.570962 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513f0378456840fcbb6a89832334f532" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:28.571002 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513f0378456840fcbb6a89832334f532" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:28.571096 3151 raft_consensus.cc:3037] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.571089 2886 raft_consensus.cc:3037] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.572042 3151 raft_consensus.cc:2454] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:28.572042 2886 raft_consensus.cc:2454] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:28.572326 3219 leader_election.cc:302] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:28.572597 3896 raft_consensus.cc:2781] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:28.572690 3896 raft_consensus.cc:686] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:28.572825 3896 consensus_queue.cc:237] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.574630 32731 catalog_manager.cc:5484] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:28.612175 3896 consensus_queue.cc:1035] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.616413 3889 consensus_queue.cc:1035] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.679661 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:28.683569 3151 raft_consensus.cc:1261] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:28.683609 2886 raft_consensus.cc:1261] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:28.684057 3889 consensus_queue.cc:1035] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.684202 3870 consensus_queue.cc:1035] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.686805 32731 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet 513f0378456840fcbb6a89832334f532 (table rebalancer_dynamic_table [id=1a46c1e427b948dcb5ca91483ae436f5]). Expected version 1 got 0
I20240502 14:00:28.687100 3894 tablet.cc:1722] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:28.687407 3894 tablet.cc:1620] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:28.687701 32730 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 513f0378456840fcbb6a89832334f532 (table rebalancer_dynamic_table [id=1a46c1e427b948dcb5ca91483ae436f5]). Expected version 1 got 0
I20240502 14:00:28.688349 32731 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 513f0378456840fcbb6a89832334f532 (table rebalancer_dynamic_table [id=1a46c1e427b948dcb5ca91483ae436f5]). Expected version 1 got 0
I20240502 14:00:28.688449 3929 tablet.cc:1722] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:28.688725 3929 tablet.cc:1620] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:28.689445 32730 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet 513f0378456840fcbb6a89832334f532 (table rebalancer_dynamic_table [id=1a46c1e427b948dcb5ca91483ae436f5]). Expected version 1 got 0
I20240502 14:00:28.689740 3865 tablet.cc:1722] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:28.690040 3865 tablet.cc:1620] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:28.693588 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=1a46c1e427b948dcb5ca91483ae436f5] alter complete (version 1)
I20240502 14:00:28.733609 3782 consensus_queue.cc:1035] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.736843 3782 consensus_queue.cc:1035] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.796716 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:28.802213 3894 tablet.cc:1722] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:28.802567 3894 tablet.cc:1620] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:28.803122 3865 tablet.cc:1722] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:28.803400 3865 tablet.cc:1620] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:28.805938 3929 tablet.cc:1722] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:28.806331 3929 tablet.cc:1620] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:28.807898 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=1a46c1e427b948dcb5ca91483ae436f5] alter complete (version 2)
I20240502 14:00:28.812717 3914 consensus_queue.cc:1035] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.819664 3914 consensus_queue.cc:1035] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.819700 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:28.826658 784 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4469ea792f4c1f9dce98e47b51f9f4 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=0b8348df8f78415b994322df0d6d1090]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.826756 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4469ea792f4c1f9dce98e47b51f9f4 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=0b8348df8f78415b994322df0d6d1090]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.826995 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4469ea792f4c1f9dce98e47b51f9f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.827014 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4469ea792f4c1f9dce98e47b51f9f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.828594 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 909bc2eb63544db0b4b3a0764be7d4a1 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=0b8348df8f78415b994322df0d6d1090]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.828621 653 tablet_service.cc:1467] Processing CreateTablet for tablet 7d23725b6cff48cbbf546394a4c54406 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=0b8348df8f78415b994322df0d6d1090]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.828860 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 909bc2eb63544db0b4b3a0764be7d4a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.828881 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d23725b6cff48cbbf546394a4c54406. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.829336 3903 tablet_bootstrap.cc:492] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:28.829530 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4469ea792f4c1f9dce98e47b51f9f4 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=0b8348df8f78415b994322df0d6d1090]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.829746 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4469ea792f4c1f9dce98e47b51f9f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.830307 3903 tablet_bootstrap.cc:654] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.831507 784 tablet_service.cc:1467] Processing CreateTablet for tablet 909bc2eb63544db0b4b3a0764be7d4a1 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=0b8348df8f78415b994322df0d6d1090]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.831507 520 tablet_service.cc:1467] Processing CreateTablet for tablet 7d23725b6cff48cbbf546394a4c54406 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=0b8348df8f78415b994322df0d6d1090]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.831737 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d23725b6cff48cbbf546394a4c54406. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.831737 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 909bc2eb63544db0b4b3a0764be7d4a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.831923 3937 tablet_bootstrap.cc:492] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:28.832115 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 7d23725b6cff48cbbf546394a4c54406 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=0b8348df8f78415b994322df0d6d1090]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.832383 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d23725b6cff48cbbf546394a4c54406. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.832904 3937 tablet_bootstrap.cc:654] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.833217 3939 tablet_bootstrap.cc:492] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:28.834053 3939 tablet_bootstrap.cc:654] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.834053 3937 tablet_bootstrap.cc:492] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:28.834131 3937 ts_tablet_manager.cc:1367] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:28.834574 3880 tablet_bootstrap.cc:492] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:28.834605 3937 raft_consensus.cc:348] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:28.834708 3937 raft_consensus.cc:374] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.834746 3937 raft_consensus.cc:729] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.834913 3940 tablet_bootstrap.cc:492] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:28.834908 3937 consensus_queue.cc:260] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:28.835160 3937 ts_tablet_manager.cc:1398] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.835317 3937 tablet_bootstrap.cc:492] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:28.835489 3880 tablet_bootstrap.cc:654] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.836241 3937 tablet_bootstrap.cc:654] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.836670 3880 tablet_bootstrap.cc:492] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:28.836752 3880 ts_tablet_manager.cc:1367] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:28.837234 3880 raft_consensus.cc:348] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.837335 3940 tablet_bootstrap.cc:654] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.837344 3880 raft_consensus.cc:374] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.837381 3880 raft_consensus.cc:729] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.837504 3880 consensus_queue.cc:260] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.837771 3880 ts_tablet_manager.cc:1398] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:28.837986 3937 tablet_bootstrap.cc:492] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:28.838057 3937 ts_tablet_manager.cc:1367] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:28.838407 3940 tablet_bootstrap.cc:492] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:28.838502 3940 ts_tablet_manager.cc:1367] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:28.838459 3937 raft_consensus.cc:348] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:28.838544 3937 raft_consensus.cc:374] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.838574 3937 raft_consensus.cc:729] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.838662 3937 consensus_queue.cc:260] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:28.838892 3937 ts_tablet_manager.cc:1398] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.838896 3940 raft_consensus.cc:348] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.839011 3940 raft_consensus.cc:374] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.839044 3940 raft_consensus.cc:729] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.839151 3940 consensus_queue.cc:260] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.839408 3940 ts_tablet_manager.cc:1398] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:28.839809 3939 tablet_bootstrap.cc:492] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:28.839917 3939 ts_tablet_manager.cc:1367] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:28.840355 3939 raft_consensus.cc:348] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:28.840489 3939 raft_consensus.cc:374] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.840528 3939 raft_consensus.cc:729] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.840646 3939 consensus_queue.cc:260] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:28.840732 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 909bc2eb63544db0b4b3a0764be7d4a1 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=0b8348df8f78415b994322df0d6d1090]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.840881 3939 ts_tablet_manager.cc:1398] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:28.841015 3939 tablet_bootstrap.cc:492] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:28.841341 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 909bc2eb63544db0b4b3a0764be7d4a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.841802 3939 tablet_bootstrap.cc:654] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.844151 3944 tablet_bootstrap.cc:492] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:28.845057 3944 tablet_bootstrap.cc:654] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.845981 3939 tablet_bootstrap.cc:492] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:28.845978 3903 tablet_bootstrap.cc:492] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:28.846062 3939 ts_tablet_manager.cc:1367] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 14:00:28.846083 3903 ts_tablet_manager.cc:1367] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 14:00:28.846482 3939 raft_consensus.cc:348] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:28.846504 3903 raft_consensus.cc:348] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:28.846583 3939 raft_consensus.cc:374] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.846591 3903 raft_consensus.cc:374] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.846619 3939 raft_consensus.cc:729] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.846625 3903 raft_consensus.cc:729] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.846732 3903 consensus_queue.cc:260] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:28.846732 3939 consensus_queue.cc:260] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:28.846978 3903 ts_tablet_manager.cc:1398] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.846987 3939 ts_tablet_manager.cc:1398] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:28.847157 3903 tablet_bootstrap.cc:492] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:28.848208 3903 tablet_bootstrap.cc:654] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.849283 3903 tablet_bootstrap.cc:492] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:28.849359 3903 ts_tablet_manager.cc:1367] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:28.849800 3903 raft_consensus.cc:348] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:28.849906 3903 raft_consensus.cc:374] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.849937 3903 raft_consensus.cc:729] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.850051 3903 consensus_queue.cc:260] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:28.850288 3903 ts_tablet_manager.cc:1398] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:28.858556 3944 tablet_bootstrap.cc:492] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:28.858646 3944 ts_tablet_manager.cc:1367] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:00:28.859094 3944 raft_consensus.cc:348] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.859194 3944 raft_consensus.cc:374] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.859230 3944 raft_consensus.cc:729] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.859354 3944 consensus_queue.cc:260] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.859632 3944 ts_tablet_manager.cc:1398] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:28.909451 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:28.914846 3894 tablet.cc:1722] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:28.915889 3929 tablet.cc:1722] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:28.916297 3865 tablet.cc:1722] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:28.920506 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=1a46c1e427b948dcb5ca91483ae436f5] alter complete (version 3)
I20240502 14:00:28.955745 3725 raft_consensus.cc:482] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:28.955883 3725 raft_consensus.cc:504] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.956374 3725 leader_election.cc:288] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:28.956681 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d23725b6cff48cbbf546394a4c54406" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:28.956756 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d23725b6cff48cbbf546394a4c54406" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:28.956823 672 raft_consensus.cc:2454] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:28.956885 3151 raft_consensus.cc:2454] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:28.957158 477 leader_election.cc:302] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:28.957369 3725 raft_consensus.cc:2781] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:28.957427 3725 raft_consensus.cc:482] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:28.957465 3725 raft_consensus.cc:3037] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.958319 3725 raft_consensus.cc:504] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.958634 3725 leader_election.cc:288] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:28.958922 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d23725b6cff48cbbf546394a4c54406" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:28.959045 672 raft_consensus.cc:3037] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.959810 672 raft_consensus.cc:2454] T 7d23725b6cff48cbbf546394a4c54406 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:28.959955 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d23725b6cff48cbbf546394a4c54406" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:28.960085 3151 raft_consensus.cc:3037] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.960809 3151 raft_consensus.cc:2454] T 7d23725b6cff48cbbf546394a4c54406 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:28.961088 477 leader_election.cc:302] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:28.961273 3725 raft_consensus.cc:2781] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:28.961378 3725 raft_consensus.cc:686] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:28.961544 3725 consensus_queue.cc:237] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:28.963462 32730 catalog_manager.cc:5484] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:29.021800 32730 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:29.022172 32730 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:29.024487 32730 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 513f0378456840fcbb6a89832334f532
I20240502 14:00:29.025101 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 513f0378456840fcbb6a89832334f532 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:29 UTC) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:29.025518 3951 tablet_replica.cc:330] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:29.025730 3951 raft_consensus.cc:2229] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:29.025929 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet 513f0378456840fcbb6a89832334f532 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:29 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:29.026034 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet 513f0378456840fcbb6a89832334f532 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:29 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:29.026321 3951 raft_consensus.cc:2258] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:29.026391 3953 tablet_replica.cc:330] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:29.026568 3953 raft_consensus.cc:2229] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:29.026772 3953 raft_consensus.cc:2258] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:29.027285 3953 ts_tablet_manager.cc:1875] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:29.027896 3951 ts_tablet_manager.cc:1875] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:29.029250 3953 ts_tablet_manager.cc:1888] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:29.029336 3953 log.cc:1196] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/513f0378456840fcbb6a89832334f532
I20240502 14:00:29.029614 3953 ts_tablet_manager.cc:1909] T 513f0378456840fcbb6a89832334f532 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:29.029955 3951 ts_tablet_manager.cc:1888] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:29.030030 3951 log.cc:1196] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/513f0378456840fcbb6a89832334f532
I20240502 14:00:29.030221 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 513f0378456840fcbb6a89832334f532 (table rebalancer_dynamic_table_new_name [id=1a46c1e427b948dcb5ca91483ae436f5]) successfully deleted
I20240502 14:00:29.030292 3951 ts_tablet_manager.cc:1909] T 513f0378456840fcbb6a89832334f532 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting consensus metadata
I20240502 14:00:29.030923 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 513f0378456840fcbb6a89832334f532 (table rebalancer_dynamic_table_new_name [id=1a46c1e427b948dcb5ca91483ae436f5]) successfully deleted
I20240502 14:00:29.032011 3952 tablet_replica.cc:330] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:29.032158 3952 raft_consensus.cc:2229] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:29.032377 3952 raft_consensus.cc:2258] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:29.033046 3952 ts_tablet_manager.cc:1875] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:29.034794 3952 ts_tablet_manager.cc:1888] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:29.034865 3952 log.cc:1196] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/513f0378456840fcbb6a89832334f532
I20240502 14:00:29.035410 3952 ts_tablet_manager.cc:1909] T 513f0378456840fcbb6a89832334f532 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:29.036069 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 513f0378456840fcbb6a89832334f532 (table rebalancer_dynamic_table_new_name [id=1a46c1e427b948dcb5ca91483ae436f5]) successfully deleted
I20240502 14:00:29.073724 3503 raft_consensus.cc:482] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:29.073856 3503 raft_consensus.cc:504] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.074311 3503 leader_election.cc:288] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:29.074736 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4469ea792f4c1f9dce98e47b51f9f4" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:29.074926 3018 raft_consensus.cc:2454] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:29.075276 2819 leader_election.cc:302] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:29.075459 3503 raft_consensus.cc:2781] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:29.075510 3503 raft_consensus.cc:482] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:29.075546 3503 raft_consensus.cc:3037] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.076568 3503 raft_consensus.cc:504] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.076936 3503 leader_election.cc:288] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:29.077028 3956 raft_consensus.cc:482] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:29.077123 3956 raft_consensus.cc:504] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:29.077333 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4469ea792f4c1f9dce98e47b51f9f4" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:29.077435 3018 raft_consensus.cc:3037] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.077508 3956 leader_election.cc:288] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:29.078485 3018 raft_consensus.cc:2454] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:29.078521 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "909bc2eb63544db0b4b3a0764be7d4a1" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:29.078646 3019 raft_consensus.cc:2454] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:29.078802 2819 leader_election.cc:302] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:29.078992 2819 leader_election.cc:302] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:29.079051 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4469ea792f4c1f9dce98e47b51f9f4" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:29.079103 806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4469ea792f4c1f9dce98e47b51f9f4" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:29.079164 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "909bc2eb63544db0b4b3a0764be7d4a1" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:29.079173 804 raft_consensus.cc:2454] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:29.079262 805 raft_consensus.cc:2454] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:29.079402 3503 raft_consensus.cc:2781] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:29.079556 3503 raft_consensus.cc:686] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:29.079643 3957 raft_consensus.cc:2781] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:29.079708 3957 raft_consensus.cc:482] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:29.079697 3503 consensus_queue.cc:237] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.079789 3957 raft_consensus.cc:3037] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.081419 3957 raft_consensus.cc:504] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:29.081617 32730 catalog_manager.cc:5484] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:29.081768 3957 leader_election.cc:288] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:29.082042 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "909bc2eb63544db0b4b3a0764be7d4a1" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:29.082063 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "909bc2eb63544db0b4b3a0764be7d4a1" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:29.082139 805 raft_consensus.cc:3037] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.082156 3019 raft_consensus.cc:3037] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.082823 3019 raft_consensus.cc:2454] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:29.083072 805 raft_consensus.cc:2454] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:29.083106 2819 leader_election.cc:302] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:29.083253 3957 raft_consensus.cc:2781] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:29.083329 3957 raft_consensus.cc:686] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:29.083483 3957 consensus_queue.cc:237] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:29.085259 32730 catalog_manager.cc:5484] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:29.127108 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:29.131991 3263 tablet_service.cc:1467] Processing CreateTablet for tablet ef38e54ae0534cdfb2221c7d2840be99 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4e16ccbad74c495e8afd92a075a13db5]), partition=
I20240502 14:00:29.131991 3130 tablet_service.cc:1467] Processing CreateTablet for tablet ef38e54ae0534cdfb2221c7d2840be99 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4e16ccbad74c495e8afd92a075a13db5]), partition=
I20240502 14:00:29.132112 2999 tablet_service.cc:1467] Processing CreateTablet for tablet ef38e54ae0534cdfb2221c7d2840be99 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4e16ccbad74c495e8afd92a075a13db5]), partition=
I20240502 14:00:29.132266 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef38e54ae0534cdfb2221c7d2840be99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.132266 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef38e54ae0534cdfb2221c7d2840be99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.132377 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef38e54ae0534cdfb2221c7d2840be99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.134575 3880 tablet_bootstrap.cc:492] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:29.134817 3960 tablet_bootstrap.cc:492] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:29.135601 3880 tablet_bootstrap.cc:654] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.135720 3960 tablet_bootstrap.cc:654] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.136932 3880 tablet_bootstrap.cc:492] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:29.137028 3880 ts_tablet_manager.cc:1367] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:29.137050 3939 tablet_bootstrap.cc:492] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:29.137364 3960 tablet_bootstrap.cc:492] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:29.137471 3960 ts_tablet_manager.cc:1367] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:29.137507 3880 raft_consensus.cc:348] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.137625 3880 raft_consensus.cc:374] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.137658 3880 raft_consensus.cc:729] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.137789 3880 consensus_queue.cc:260] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.138036 3939 tablet_bootstrap.cc:654] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.138259 3880 ts_tablet_manager.cc:1398] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:29.138300 3960 raft_consensus.cc:348] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.138407 3960 raft_consensus.cc:374] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.138438 3960 raft_consensus.cc:729] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.138584 3960 consensus_queue.cc:260] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.138919 3960 ts_tablet_manager.cc:1398] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:29.139204 3939 tablet_bootstrap.cc:492] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:29.139294 3939 ts_tablet_manager.cc:1367] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:29.139647 3939 raft_consensus.cc:348] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.139765 3939 raft_consensus.cc:374] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.139811 3939 raft_consensus.cc:729] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.139964 3939 consensus_queue.cc:260] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.140209 3939 ts_tablet_manager.cc:1398] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:29.224476 3934 raft_consensus.cc:482] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:29.224622 3934 raft_consensus.cc:504] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.225082 3934 leader_election.cc:288] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:29.225404 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef38e54ae0534cdfb2221c7d2840be99" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:29.225497 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef38e54ae0534cdfb2221c7d2840be99" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:29.225553 3283 raft_consensus.cc:2454] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:29.225634 3151 raft_consensus.cc:2454] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:29.225859 2952 leader_election.cc:302] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:29.226039 3934 raft_consensus.cc:2781] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:29.226102 3934 raft_consensus.cc:482] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:29.226161 3934 raft_consensus.cc:3037] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.227156 3934 raft_consensus.cc:504] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.227468 3934 leader_election.cc:288] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:29.227761 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef38e54ae0534cdfb2221c7d2840be99" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:29.227864 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef38e54ae0534cdfb2221c7d2840be99" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:29.227912 3283 raft_consensus.cc:3037] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.227979 3151 raft_consensus.cc:3037] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.228667 3283 raft_consensus.cc:2454] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:29.228768 3151 raft_consensus.cc:2454] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:29.228926 2952 leader_election.cc:302] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:29.229152 3934 raft_consensus.cc:2781] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:29.229226 3934 raft_consensus.cc:686] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:29.229364 3934 consensus_queue.cc:237] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.231120 32732 catalog_manager.cc:5484] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:29.355713 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:29.359946 3151 raft_consensus.cc:1261] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.360353 3283 raft_consensus.cc:1261] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.360445 3934 consensus_queue.cc:1035] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.360911 3914 consensus_queue.cc:1035] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.361779 3725 consensus_queue.cc:1035] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.363457 32732 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet ef38e54ae0534cdfb2221c7d2840be99 (table rebalancer_dynamic_table [id=4e16ccbad74c495e8afd92a075a13db5]). Expected version 1 got 0
I20240502 14:00:29.364251 32733 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet ef38e54ae0534cdfb2221c7d2840be99 (table rebalancer_dynamic_table [id=4e16ccbad74c495e8afd92a075a13db5]). Expected version 1 got 0
I20240502 14:00:29.364470 3894 tablet.cc:1722] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:29.364780 3894 tablet.cc:1620] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:29.364806 3970 tablet.cc:1722] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:29.365101 3970 tablet.cc:1620] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:29.365392 32733 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet ef38e54ae0534cdfb2221c7d2840be99 (table rebalancer_dynamic_table [id=4e16ccbad74c495e8afd92a075a13db5]). Expected version 1 got 0
I20240502 14:00:29.365796 3865 tablet.cc:1722] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:29.366089 3865 tablet.cc:1620] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:29.369434 3926 consensus_queue.cc:1035] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.372258 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=4e16ccbad74c495e8afd92a075a13db5] alter complete (version 1)
I20240502 14:00:29.468793 804 raft_consensus.cc:3037] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.470314 3957 consensus_queue.cc:1035] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.473855 3957 consensus_queue.cc:1035] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.474395 3957 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.475953 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:29.481542 3970 tablet.cc:1722] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:29.481879 3970 tablet.cc:1620] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:29.482596 3503 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.485395 3894 tablet.cc:1722] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:29.485709 3894 tablet.cc:1620] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:29.486042 3865 tablet.cc:1722] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:29.486348 3865 tablet.cc:1620] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:29.486637 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=4e16ccbad74c495e8afd92a075a13db5] alter complete (version 2)
I20240502 14:00:29.590503 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:29.596360 3970 tablet.cc:1722] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:29.596912 3894 tablet.cc:1722] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:29.597113 3865 tablet.cc:1722] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:29.601325 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=4e16ccbad74c495e8afd92a075a13db5] alter complete (version 3)
I20240502 14:00:29.638039 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:29.644896 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 6451a34c3fb34207bcd761556b0e08dd (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=c2a7fffdc4d148a5839276762398bfd9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.645259 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6451a34c3fb34207bcd761556b0e08dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.645488 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 58a0cf9cd92a43ca8c481964707eda6d (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=c2a7fffdc4d148a5839276762398bfd9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.645632 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 58a0cf9cd92a43ca8c481964707eda6d (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=c2a7fffdc4d148a5839276762398bfd9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.645756 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58a0cf9cd92a43ca8c481964707eda6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.645915 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58a0cf9cd92a43ca8c481964707eda6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.648162 3984 tablet_bootstrap.cc:492] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:29.648211 653 tablet_service.cc:1467] Processing CreateTablet for tablet 41aea66376974d36978dfa059d52aa79 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=c2a7fffdc4d148a5839276762398bfd9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.648521 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41aea66376974d36978dfa059d52aa79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.648636 3983 tablet_bootstrap.cc:492] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:29.648715 388 tablet_service.cc:1467] Processing CreateTablet for tablet 6451a34c3fb34207bcd761556b0e08dd (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=c2a7fffdc4d148a5839276762398bfd9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.648993 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6451a34c3fb34207bcd761556b0e08dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.649574 3983 tablet_bootstrap.cc:654] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.649866 3984 tablet_bootstrap.cc:654] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.650955 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 41aea66376974d36978dfa059d52aa79 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=c2a7fffdc4d148a5839276762398bfd9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.651219 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41aea66376974d36978dfa059d52aa79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.651244 3984 tablet_bootstrap.cc:492] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:29.651342 3984 ts_tablet_manager.cc:1367] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:29.651898 3984 raft_consensus.cc:348] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.652043 3984 raft_consensus.cc:374] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.652099 3984 raft_consensus.cc:729] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.652133 3983 tablet_bootstrap.cc:492] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:29.652223 3983 ts_tablet_manager.cc:1367] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:29.652248 3984 consensus_queue.cc:260] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.652727 3984 ts_tablet_manager.cc:1398] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.652691 3983 raft_consensus.cc:348] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.652820 3983 raft_consensus.cc:374] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.652864 3983 raft_consensus.cc:729] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.653010 3983 consensus_queue.cc:260] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.653352 3983 ts_tablet_manager.cc:1398] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:29.653667 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 6451a34c3fb34207bcd761556b0e08dd (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=c2a7fffdc4d148a5839276762398bfd9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.653950 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6451a34c3fb34207bcd761556b0e08dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.653128 3988 tablet_bootstrap.cc:492] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:29.655161 3988 tablet_bootstrap.cc:654] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.656610 3988 tablet_bootstrap.cc:492] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:29.656718 3988 ts_tablet_manager.cc:1367] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:29.657140 3988 raft_consensus.cc:348] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.657249 3988 raft_consensus.cc:374] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.657298 3988 raft_consensus.cc:729] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.657438 3988 consensus_queue.cc:260] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.657462 520 tablet_service.cc:1467] Processing CreateTablet for tablet 41aea66376974d36978dfa059d52aa79 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=c2a7fffdc4d148a5839276762398bfd9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.657488 522 tablet_service.cc:1467] Processing CreateTablet for tablet 58a0cf9cd92a43ca8c481964707eda6d (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=c2a7fffdc4d148a5839276762398bfd9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.657723 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41aea66376974d36978dfa059d52aa79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.658175 3988 ts_tablet_manager.cc:1398] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.660604 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58a0cf9cd92a43ca8c481964707eda6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.663542 3989 tablet_bootstrap.cc:492] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:29.664016 3985 tablet_bootstrap.cc:492] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:29.664518 3989 tablet_bootstrap.cc:654] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.665149 3985 tablet_bootstrap.cc:654] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.665690 3989 tablet_bootstrap.cc:492] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:29.665781 3989 ts_tablet_manager.cc:1367] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:29.666204 3989 raft_consensus.cc:348] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.666313 3989 raft_consensus.cc:374] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.666342 3989 raft_consensus.cc:729] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.666466 3989 consensus_queue.cc:260] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.666846 3989 ts_tablet_manager.cc:1398] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.667567 3985 tablet_bootstrap.cc:492] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:29.667699 3985 ts_tablet_manager.cc:1367] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:29.668232 3985 raft_consensus.cc:348] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.668371 3985 raft_consensus.cc:374] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.668452 3985 raft_consensus.cc:729] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.668596 3985 consensus_queue.cc:260] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.668853 3985 ts_tablet_manager.cc:1398] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:29.669026 3985 tablet_bootstrap.cc:492] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:29.669934 3985 tablet_bootstrap.cc:654] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.671082 3985 tablet_bootstrap.cc:492] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:29.671195 3985 ts_tablet_manager.cc:1367] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:29.671813 3985 raft_consensus.cc:348] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.672000 3985 raft_consensus.cc:374] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.672047 3985 raft_consensus.cc:729] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.672173 3985 consensus_queue.cc:260] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.672477 3985 ts_tablet_manager.cc:1398] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.676023 3990 tablet_bootstrap.cc:492] T 41aea66376974d36978dfa059d52aa79 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:29.676098 3992 tablet_bootstrap.cc:492] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:29.677179 3992 tablet_bootstrap.cc:654] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.677186 3990 tablet_bootstrap.cc:654] T 41aea66376974d36978dfa059d52aa79 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.678421 3990 tablet_bootstrap.cc:492] T 41aea66376974d36978dfa059d52aa79 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:29.678520 3990 ts_tablet_manager.cc:1367] T 41aea66376974d36978dfa059d52aa79 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:29.678632 3992 tablet_bootstrap.cc:492] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:29.678725 3992 ts_tablet_manager.cc:1367] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:29.679056 3990 raft_consensus.cc:348] T 41aea66376974d36978dfa059d52aa79 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.679183 3990 raft_consensus.cc:374] T 41aea66376974d36978dfa059d52aa79 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.679256 3990 raft_consensus.cc:729] T 41aea66376974d36978dfa059d52aa79 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.679121 3992 raft_consensus.cc:348] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.679332 3992 raft_consensus.cc:374] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.679322 3503 raft_consensus.cc:482] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:29.679401 3992 raft_consensus.cc:729] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.679420 3503 raft_consensus.cc:504] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.679430 3990 consensus_queue.cc:260] T 41aea66376974d36978dfa059d52aa79 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.679560 3992 consensus_queue.cc:260] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.679912 3503 leader_election.cc:288] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:29.680215 3990 ts_tablet_manager.cc:1398] T 41aea66376974d36978dfa059d52aa79 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:29.680704 3992 ts_tablet_manager.cc:1398] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:29.680677 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a0cf9cd92a43ca8c481964707eda6d" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:29.680792 3018 raft_consensus.cc:2454] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:29.680858 3992 tablet_bootstrap.cc:492] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:29.681237 2819 leader_election.cc:302] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:29.681494 3503 raft_consensus.cc:2781] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:29.681548 3503 raft_consensus.cc:482] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:29.681591 3503 raft_consensus.cc:3037] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.681763 3992 tablet_bootstrap.cc:654] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.682636 3503 raft_consensus.cc:504] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.682963 3503 leader_election.cc:288] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:29.683333 3992 tablet_bootstrap.cc:492] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:29.683425 3992 ts_tablet_manager.cc:1367] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:29.683485 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a0cf9cd92a43ca8c481964707eda6d" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:29.683584 3018 raft_consensus.cc:3037] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.684003 3992 raft_consensus.cc:348] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.684092 3992 raft_consensus.cc:374] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.684125 3992 raft_consensus.cc:729] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.684278 3992 consensus_queue.cc:260] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.684630 3992 ts_tablet_manager.cc:1398] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.684717 3018 raft_consensus.cc:2454] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:29.685025 2819 leader_election.cc:302] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:29.685458 3956 raft_consensus.cc:2781] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:29.685565 3956 raft_consensus.cc:686] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:29.685720 3956 consensus_queue.cc:237] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.687481 32733 catalog_manager.cc:5484] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:29.692734 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a0cf9cd92a43ca8c481964707eda6d" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:29.692869 541 raft_consensus.cc:2454] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:29.693071 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a0cf9cd92a43ca8c481964707eda6d" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:29.693159 540 raft_consensus.cc:3037] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.694294 540 raft_consensus.cc:2454] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:29.702926 3926 raft_consensus.cc:482] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:29.703063 3926 raft_consensus.cc:504] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.703549 3926 leader_election.cc:288] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:29.703591 32733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:29.703716 32733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:29.703872 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41aea66376974d36978dfa059d52aa79" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:29.704011 672 raft_consensus.cc:2454] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:29.704541 474 leader_election.cc:302] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:29.704710 3926 raft_consensus.cc:2781] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:29.704768 3926 raft_consensus.cc:482] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:29.704818 3926 raft_consensus.cc:3037] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.705765 3926 raft_consensus.cc:504] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.706179 3926 leader_election.cc:288] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:29.706670 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet ef38e54ae0534cdfb2221c7d2840be99
I20240502 14:00:29.706991 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41aea66376974d36978dfa059d52aa79" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:29.707160 672 raft_consensus.cc:3037] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.707402 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet ef38e54ae0534cdfb2221c7d2840be99 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:29 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:29.707727 4004 tablet_replica.cc:330] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:29.707914 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet ef38e54ae0534cdfb2221c7d2840be99 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:29 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:29.708040 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet ef38e54ae0534cdfb2221c7d2840be99 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:29 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:29.708257 672 raft_consensus.cc:2454] T 41aea66376974d36978dfa059d52aa79 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:29.708464 4006 tablet_replica.cc:330] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:29.708580 474 leader_election.cc:302] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:29.708595 4005 tablet_replica.cc:330] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:29.708712 4006 raft_consensus.cc:2229] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:29.708743 3926 raft_consensus.cc:2781] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:29.708738 4005 raft_consensus.cc:2229] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:29.708818 3926 raft_consensus.cc:686] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:29.708889 4005 raft_consensus.cc:2258] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:29.708940 3926 consensus_queue.cc:237] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.709026 4006 raft_consensus.cc:2258] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:29.709544 4005 ts_tablet_manager.cc:1875] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:29.709616 4006 ts_tablet_manager.cc:1875] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:29.708330 4004 raft_consensus.cc:2229] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:29.709540 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41aea66376974d36978dfa059d52aa79" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:29.710348 3284 raft_consensus.cc:3037] T 41aea66376974d36978dfa059d52aa79 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.709672 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41aea66376974d36978dfa059d52aa79" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:29.711063 32733 catalog_manager.cc:5484] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:29.711511 3284 raft_consensus.cc:2454] T 41aea66376974d36978dfa059d52aa79 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:29.711752 4004 raft_consensus.cc:2258] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:29.711922 4005 ts_tablet_manager.cc:1888] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:29.711989 4005 log.cc:1196] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/ef38e54ae0534cdfb2221c7d2840be99
I20240502 14:00:29.712314 4005 ts_tablet_manager.cc:1909] T ef38e54ae0534cdfb2221c7d2840be99 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:29.712630 4004 ts_tablet_manager.cc:1875] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:29.713018 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet ef38e54ae0534cdfb2221c7d2840be99 (table rebalancer_dynamic_table_new_name [id=4e16ccbad74c495e8afd92a075a13db5]) successfully deleted
I20240502 14:00:29.714624 4006 ts_tablet_manager.cc:1888] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:29.714709 4006 log.cc:1196] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/ef38e54ae0534cdfb2221c7d2840be99
I20240502 14:00:29.714953 4004 ts_tablet_manager.cc:1888] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:29.715018 4006 ts_tablet_manager.cc:1909] T ef38e54ae0534cdfb2221c7d2840be99 P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:29.715047 4004 log.cc:1196] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/ef38e54ae0534cdfb2221c7d2840be99
I20240502 14:00:29.715293 4004 ts_tablet_manager.cc:1909] T ef38e54ae0534cdfb2221c7d2840be99 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:29.715766 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet ef38e54ae0534cdfb2221c7d2840be99 (table rebalancer_dynamic_table_new_name [id=4e16ccbad74c495e8afd92a075a13db5]) successfully deleted
I20240502 14:00:29.716007 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet ef38e54ae0534cdfb2221c7d2840be99 (table rebalancer_dynamic_table_new_name [id=4e16ccbad74c495e8afd92a075a13db5]) successfully deleted
I20240502 14:00:29.808943 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:29.814313 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 3f01fef69d4c4d9fa927e426e679b177 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=653bbcca2f784d8aa5d5bf0f04cdf22c]), partition=
I20240502 14:00:29.814342 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 3f01fef69d4c4d9fa927e426e679b177 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=653bbcca2f784d8aa5d5bf0f04cdf22c]), partition=
I20240502 14:00:29.814586 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f01fef69d4c4d9fa927e426e679b177. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.814635 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f01fef69d4c4d9fa927e426e679b177. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.816494 3990 tablet_bootstrap.cc:492] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:29.816866 784 tablet_service.cc:1467] Processing CreateTablet for tablet 3f01fef69d4c4d9fa927e426e679b177 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=653bbcca2f784d8aa5d5bf0f04cdf22c]), partition=
I20240502 14:00:29.817099 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f01fef69d4c4d9fa927e426e679b177. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.817571 3990 tablet_bootstrap.cc:654] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.819049 3990 tablet_bootstrap.cc:492] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:29.819132 3990 ts_tablet_manager.cc:1367] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:29.819183 4009 tablet_bootstrap.cc:492] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:29.819216 3983 tablet_bootstrap.cc:492] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:29.819614 3990 raft_consensus.cc:348] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.819739 3990 raft_consensus.cc:374] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.819772 3990 raft_consensus.cc:729] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.819921 3990 consensus_queue.cc:260] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.820111 3983 tablet_bootstrap.cc:654] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.820132 4009 tablet_bootstrap.cc:654] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.820233 3990 ts_tablet_manager.cc:1398] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.822000 3983 tablet_bootstrap.cc:492] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:29.822094 3983 ts_tablet_manager.cc:1367] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:29.823161 4009 tablet_bootstrap.cc:492] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:29.823310 4009 ts_tablet_manager.cc:1367] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:29.822563 3983 raft_consensus.cc:348] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.823563 3983 raft_consensus.cc:374] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.823613 3983 raft_consensus.cc:729] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.823731 3983 consensus_queue.cc:260] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.823851 4009 raft_consensus.cc:348] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.823959 4009 raft_consensus.cc:374] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.823992 4009 raft_consensus.cc:729] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.824059 3983 ts_tablet_manager.cc:1398] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:29.824113 4009 consensus_queue.cc:260] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.824443 4009 ts_tablet_manager.cc:1398] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.889168 3769 raft_consensus.cc:482] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:29.889290 3769 raft_consensus.cc:504] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.889680 3769 leader_election.cc:288] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:29.890134 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f01fef69d4c4d9fa927e426e679b177" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:29.890295 3284 raft_consensus.cc:2454] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:29.890600 739 leader_election.cc:302] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:29.890774 3769 raft_consensus.cc:2781] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:29.890846 3769 raft_consensus.cc:482] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:29.890892 3769 raft_consensus.cc:3037] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.891858 3769 raft_consensus.cc:504] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.892201 3769 leader_election.cc:288] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:29.892567 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f01fef69d4c4d9fa927e426e679b177" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:29.892670 3284 raft_consensus.cc:3037] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.893509 3284 raft_consensus.cc:2454] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:29.893854 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f01fef69d4c4d9fa927e426e679b177" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:29.893998 3151 raft_consensus.cc:2454] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:29.894029 739 leader_election.cc:302] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:29.894183 3769 raft_consensus.cc:2781] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:29.894228 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f01fef69d4c4d9fa927e426e679b177" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:29.894333 3152 raft_consensus.cc:3037] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.894377 3769 raft_consensus.cc:686] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:29.894527 3769 consensus_queue.cc:237] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:29.895241 3152 raft_consensus.cc:2454] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:29.896926 32733 catalog_manager.cc:5484] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:29.976475 3968 raft_consensus.cc:482] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:29.976610 3968 raft_consensus.cc:504] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.976984 3968 leader_election.cc:288] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:29.977310 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6451a34c3fb34207bcd761556b0e08dd" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:29.977361 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6451a34c3fb34207bcd761556b0e08dd" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:29.977442 405 raft_consensus.cc:2454] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:29.977483 3152 raft_consensus.cc:2454] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:29.977752 2954 leader_election.cc:302] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:29.977921 3968 raft_consensus.cc:2781] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:29.977994 3968 raft_consensus.cc:482] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:29.978034 3968 raft_consensus.cc:3037] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.979032 3968 raft_consensus.cc:504] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.979408 3968 leader_election.cc:288] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:29.979801 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6451a34c3fb34207bcd761556b0e08dd" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:29.979800 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6451a34c3fb34207bcd761556b0e08dd" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:29.979951 3152 raft_consensus.cc:3037] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.979950 405 raft_consensus.cc:3037] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.980942 3152 raft_consensus.cc:2454] T 6451a34c3fb34207bcd761556b0e08dd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:29.980942 405 raft_consensus.cc:2454] T 6451a34c3fb34207bcd761556b0e08dd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:29.981341 2954 leader_election.cc:302] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:29.981557 3968 raft_consensus.cc:2781] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:29.981655 3968 raft_consensus.cc:686] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:29.981801 3968 consensus_queue.cc:237] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:29.983738 32733 catalog_manager.cc:5484] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:30.014814 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:30.019145 3152 raft_consensus.cc:1261] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.019145 3284 raft_consensus.cc:1261] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.019649 4015 consensus_queue.cc:1035] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.019951 3769 consensus_queue.cc:1035] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.026660 32730 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 3f01fef69d4c4d9fa927e426e679b177 (table rebalancer_dynamic_table [id=653bbcca2f784d8aa5d5bf0f04cdf22c]). Expected version 1 got 0
I20240502 14:00:30.026984 32733 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 3f01fef69d4c4d9fa927e426e679b177 (table rebalancer_dynamic_table [id=653bbcca2f784d8aa5d5bf0f04cdf22c]). Expected version 1 got 0
I20240502 14:00:30.029302 4022 tablet.cc:1722] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:30.030391 4022 tablet.cc:1620] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:30.030501 3894 tablet.cc:1722] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:30.030783 3894 tablet.cc:1620] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:30.032351 32730 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet 3f01fef69d4c4d9fa927e426e679b177 (table rebalancer_dynamic_table [id=653bbcca2f784d8aa5d5bf0f04cdf22c]). Expected version 1 got 0
I20240502 14:00:30.033319 3865 tablet.cc:1722] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:30.033613 3865 tablet.cc:1620] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:30.033833 32733 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 3f01fef69d4c4d9fa927e426e679b177 (table rebalancer_dynamic_table [id=653bbcca2f784d8aa5d5bf0f04cdf22c]). Expected version 1 got 0
I20240502 14:00:30.038959 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=653bbcca2f784d8aa5d5bf0f04cdf22c] alter complete (version 1)
I20240502 14:00:30.124593 3956 consensus_queue.cc:1035] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.128134 3957 consensus_queue.cc:1035] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.142263 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:30.148026 4022 tablet.cc:1722] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:30.148361 4022 tablet.cc:1620] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:30.148422 3865 tablet.cc:1722] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:30.148716 3865 tablet.cc:1620] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:30.149341 3894 tablet.cc:1722] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:30.149636 3894 tablet.cc:1620] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:30.153941 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=653bbcca2f784d8aa5d5bf0f04cdf22c] alter complete (version 2)
I20240502 14:00:30.216236 4012 consensus_queue.cc:1035] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.219216 3725 consensus_queue.cc:1035] T 41aea66376974d36978dfa059d52aa79 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.256132 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:30.261202 4022 tablet.cc:1722] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:30.261592 3865 tablet.cc:1722] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:30.263695 3894 tablet.cc:1722] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:30.266130 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=653bbcca2f784d8aa5d5bf0f04cdf22c] alter complete (version 3)
I20240502 14:00:30.368161 32732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:30.368319 32732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:30.370322 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 3f01fef69d4c4d9fa927e426e679b177
I20240502 14:00:30.370918 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f01fef69d4c4d9fa927e426e679b177 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:30 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:30.370972 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f01fef69d4c4d9fa927e426e679b177 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:30 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:30.371237 4031 tablet_replica.cc:330] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:30.371259 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f01fef69d4c4d9fa927e426e679b177 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:30 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:30.371310 4032 tablet_replica.cc:330] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:30.371424 4032 raft_consensus.cc:2229] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:30.371439 4031 raft_consensus.cc:2229] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:30.371560 4031 raft_consensus.cc:2258] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:30.371554 4033 tablet_replica.cc:330] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:30.371654 4032 raft_consensus.cc:2258] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:30.371706 4033 raft_consensus.cc:2229] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:30.371906 4033 raft_consensus.cc:2258] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:30.372373 4031 ts_tablet_manager.cc:1875] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:30.372448 4032 ts_tablet_manager.cc:1875] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:30.372529 4033 ts_tablet_manager.cc:1875] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:30.374804 4032 ts_tablet_manager.cc:1888] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:30.374804 4031 ts_tablet_manager.cc:1888] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:30.374890 4031 log.cc:1196] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/3f01fef69d4c4d9fa927e426e679b177
I20240502 14:00:30.374890 4032 log.cc:1196] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/3f01fef69d4c4d9fa927e426e679b177
I20240502 14:00:30.375238 4031 ts_tablet_manager.cc:1909] T 3f01fef69d4c4d9fa927e426e679b177 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:30.375238 4032 ts_tablet_manager.cc:1909] T 3f01fef69d4c4d9fa927e426e679b177 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:30.375833 4033 ts_tablet_manager.cc:1888] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:30.375936 4033 log.cc:1196] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/3f01fef69d4c4d9fa927e426e679b177
I20240502 14:00:30.375918 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 3f01fef69d4c4d9fa927e426e679b177 (table rebalancer_dynamic_table_new_name [id=653bbcca2f784d8aa5d5bf0f04cdf22c]) successfully deleted
I20240502 14:00:30.376101 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 3f01fef69d4c4d9fa927e426e679b177 (table rebalancer_dynamic_table_new_name [id=653bbcca2f784d8aa5d5bf0f04cdf22c]) successfully deleted
I20240502 14:00:30.376224 4033 ts_tablet_manager.cc:1909] T 3f01fef69d4c4d9fa927e426e679b177 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:30.376855 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 3f01fef69d4c4d9fa927e426e679b177 (table rebalancer_dynamic_table_new_name [id=653bbcca2f784d8aa5d5bf0f04cdf22c]) successfully deleted
I20240502 14:00:30.421789 3934 consensus_queue.cc:1035] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.425997 4035 consensus_queue.cc:1035] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.472327 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:30.477849 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 07610855449c4cd7a9ee4a99c477f73a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9030555d09c0495b8113952059600b46]), partition=
I20240502 14:00:30.477978 653 tablet_service.cc:1467] Processing CreateTablet for tablet 07610855449c4cd7a9ee4a99c477f73a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9030555d09c0495b8113952059600b46]), partition=
I20240502 14:00:30.478085 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 07610855449c4cd7a9ee4a99c477f73a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9030555d09c0495b8113952059600b46]), partition=
I20240502 14:00:30.478176 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07610855449c4cd7a9ee4a99c477f73a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.478221 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07610855449c4cd7a9ee4a99c477f73a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.478333 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07610855449c4cd7a9ee4a99c477f73a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.480365 4038 tablet_bootstrap.cc:492] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:30.480885 4039 tablet_bootstrap.cc:492] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:30.481453 4038 tablet_bootstrap.cc:654] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.482017 4039 tablet_bootstrap.cc:654] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.482450 4038 tablet_bootstrap.cc:492] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:30.482556 4038 ts_tablet_manager.cc:1367] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:30.482643 4040 tablet_bootstrap.cc:492] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:30.483011 4038 raft_consensus.cc:348] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.483227 4038 raft_consensus.cc:374] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.483325 4038 raft_consensus.cc:729] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.483402 4040 tablet_bootstrap.cc:654] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.483573 4038 consensus_queue.cc:260] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.484138 4038 ts_tablet_manager.cc:1398] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:30.484710 4040 tablet_bootstrap.cc:492] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:30.484822 4040 ts_tablet_manager.cc:1367] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 14:00:30.485209 4040 raft_consensus.cc:348] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.485342 4040 raft_consensus.cc:374] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.485402 4040 raft_consensus.cc:729] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.485540 4040 consensus_queue.cc:260] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.485827 4040 ts_tablet_manager.cc:1398] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:30.491102 4039 tablet_bootstrap.cc:492] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:30.491209 4039 ts_tablet_manager.cc:1367] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:30.491703 4039 raft_consensus.cc:348] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.491842 4039 raft_consensus.cc:374] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.491886 4039 raft_consensus.cc:729] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.492010 4039 consensus_queue.cc:260] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.492218 4039 ts_tablet_manager.cc:1398] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:30.515058 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:30.521262 388 tablet_service.cc:1467] Processing CreateTablet for tablet ec443abbb4954bf3b28336e0b5bc0c4b (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.521620 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec443abbb4954bf3b28336e0b5bc0c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.521749 2999 tablet_service.cc:1467] Processing CreateTablet for tablet ec443abbb4954bf3b28336e0b5bc0c4b (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.522096 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec443abbb4954bf3b28336e0b5bc0c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.522753 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 6346d5ea09834bddb483da26f9bce936 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.522745 386 tablet_service.cc:1467] Processing CreateTablet for tablet caffa941c89c48d1b3d36ca47dc49eb2 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.522974 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caffa941c89c48d1b3d36ca47dc49eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.522982 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6346d5ea09834bddb483da26f9bce936. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.524080 2997 tablet_service.cc:1467] Processing CreateTablet for tablet caffa941c89c48d1b3d36ca47dc49eb2 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.524552 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 6346d5ea09834bddb483da26f9bce936 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.524794 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6346d5ea09834bddb483da26f9bce936. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.525684 4045 tablet_bootstrap.cc:492] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:30.525781 2866 tablet_service.cc:1467] Processing CreateTablet for tablet ec443abbb4954bf3b28336e0b5bc0c4b (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.525838 2865 tablet_service.cc:1467] Processing CreateTablet for tablet 6346d5ea09834bddb483da26f9bce936 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.525858 2864 tablet_service.cc:1467] Processing CreateTablet for tablet caffa941c89c48d1b3d36ca47dc49eb2 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.526103 2865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6346d5ea09834bddb483da26f9bce936. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.526664 4045 tablet_bootstrap.cc:654] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.527931 4045 tablet_bootstrap.cc:492] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:30.528026 4045 ts_tablet_manager.cc:1367] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:30.528427 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caffa941c89c48d1b3d36ca47dc49eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.528540 4045 raft_consensus.cc:348] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:30.528642 4045 raft_consensus.cc:374] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.528671 4045 raft_consensus.cc:729] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.528781 4045 consensus_queue.cc:260] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:30.529039 4045 ts_tablet_manager.cc:1398] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.529227 4045 tablet_bootstrap.cc:492] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:30.531276 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caffa941c89c48d1b3d36ca47dc49eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.531399 4047 tablet_bootstrap.cc:492] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:30.531400 4045 tablet_bootstrap.cc:654] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.532315 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec443abbb4954bf3b28336e0b5bc0c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.532442 4047 tablet_bootstrap.cc:654] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.534034 4038 tablet_bootstrap.cc:492] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:30.535297 4045 tablet_bootstrap.cc:492] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:30.535387 4045 ts_tablet_manager.cc:1367] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:30.535395 4047 tablet_bootstrap.cc:492] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:30.535476 4047 ts_tablet_manager.cc:1367] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:30.535864 4045 raft_consensus.cc:348] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:30.535899 4047 raft_consensus.cc:348] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.535982 4047 raft_consensus.cc:374] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.535982 4045 raft_consensus.cc:374] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.536016 4047 raft_consensus.cc:729] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.536016 4045 raft_consensus.cc:729] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.536146 4045 consensus_queue.cc:260] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:30.536163 4047 consensus_queue.cc:260] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.536437 4045 ts_tablet_manager.cc:1398] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.536515 4047 ts_tablet_manager.cc:1398] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:30.536696 4047 tablet_bootstrap.cc:492] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:30.536854 4050 tablet_bootstrap.cc:492] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:30.537817 4050 tablet_bootstrap.cc:654] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.538509 4047 tablet_bootstrap.cc:654] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.539227 4050 tablet_bootstrap.cc:492] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:30.539323 4050 ts_tablet_manager.cc:1367] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:30.536281 4038 tablet_bootstrap.cc:654] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.539789 4050 raft_consensus.cc:348] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:30.539958 4050 raft_consensus.cc:374] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.540144 4050 raft_consensus.cc:729] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.540266 4050 consensus_queue.cc:260] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:30.540704 4050 ts_tablet_manager.cc:1398] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:30.540915 4038 tablet_bootstrap.cc:492] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:30.540935 4050 tablet_bootstrap.cc:492] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:30.541013 4038 ts_tablet_manager.cc:1367] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:30.541471 4047 tablet_bootstrap.cc:492] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:30.541550 4047 ts_tablet_manager.cc:1367] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:30.541520 4038 raft_consensus.cc:348] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.541618 4038 raft_consensus.cc:374] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.541656 4038 raft_consensus.cc:729] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.541800 4038 consensus_queue.cc:260] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.541970 4047 raft_consensus.cc:348] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:30.542057 4047 raft_consensus.cc:374] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.542064 4038 ts_tablet_manager.cc:1398] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:30.542093 4047 raft_consensus.cc:729] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.542173 4050 tablet_bootstrap.cc:654] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.542210 4047 consensus_queue.cc:260] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:30.542464 4047 ts_tablet_manager.cc:1398] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:30.542609 4047 tablet_bootstrap.cc:492] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:30.543359 4047 tablet_bootstrap.cc:654] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.544864 4047 tablet_bootstrap.cc:492] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:30.544945 4047 ts_tablet_manager.cc:1367] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:30.545074 4050 tablet_bootstrap.cc:492] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:30.545158 4050 ts_tablet_manager.cc:1367] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:30.545396 4047 raft_consensus.cc:348] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:30.545491 4047 raft_consensus.cc:374] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.545522 4047 raft_consensus.cc:729] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.545635 4047 consensus_queue.cc:260] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:30.545714 4050 raft_consensus.cc:348] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:30.545810 4050 raft_consensus.cc:374] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.545846 4050 raft_consensus.cc:729] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.545884 4047 ts_tablet_manager.cc:1398] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:30.545981 4050 consensus_queue.cc:260] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:30.546213 4050 ts_tablet_manager.cc:1398] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:30.546386 4050 tablet_bootstrap.cc:492] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:30.547238 4050 tablet_bootstrap.cc:654] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.548398 4050 tablet_bootstrap.cc:492] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:30.548486 4050 ts_tablet_manager.cc:1367] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:30.548939 4050 raft_consensus.cc:348] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.549060 4050 raft_consensus.cc:374] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.549100 4050 raft_consensus.cc:729] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.549232 4050 consensus_queue.cc:260] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.549527 4050 ts_tablet_manager.cc:1398] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.567026 3748 raft_consensus.cc:482] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:30.567145 3748 raft_consensus.cc:504] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:30.567566 3748 leader_election.cc:288] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:30.571748 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caffa941c89c48d1b3d36ca47dc49eb2" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:30.571933 2886 raft_consensus.cc:2454] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:30.572239 344 leader_election.cc:302] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:30.572420 3748 raft_consensus.cc:2781] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:30.572511 3748 raft_consensus.cc:482] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:30.572474 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caffa941c89c48d1b3d36ca47dc49eb2" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:30.572559 3748 raft_consensus.cc:3037] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.572604 3018 raft_consensus.cc:2454] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:30.573561 3748 raft_consensus.cc:504] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:30.573868 3748 leader_election.cc:288] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:30.574172 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caffa941c89c48d1b3d36ca47dc49eb2" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:30.574285 3018 raft_consensus.cc:3037] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.575076 3018 raft_consensus.cc:2454] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:30.575328 341 leader_election.cc:302] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:30.575313 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caffa941c89c48d1b3d36ca47dc49eb2" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:30.575424 2886 raft_consensus.cc:3037] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.576315 2886 raft_consensus.cc:2454] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:30.576495 3748 raft_consensus.cc:2781] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:30.576715 3748 raft_consensus.cc:686] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:30.576872 3748 consensus_queue.cc:237] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:30.578156 3870 raft_consensus.cc:482] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:30.578263 3870 raft_consensus.cc:504] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.578663 3870 leader_election.cc:288] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:30.578953 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6346d5ea09834bddb483da26f9bce936" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:30.579052 2886 raft_consensus.cc:2454] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:30.579150 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6346d5ea09834bddb483da26f9bce936" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:30.579260 3018 raft_consensus.cc:2454] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:30.579496 3219 leader_election.cc:302] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:30.580366 32730 catalog_manager.cc:5484] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:30.580519 3870 raft_consensus.cc:2781] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:30.580571 3870 raft_consensus.cc:482] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:30.580617 3870 raft_consensus.cc:3037] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.581518 3870 raft_consensus.cc:504] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.581853 3870 leader_election.cc:288] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:30.582581 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6346d5ea09834bddb483da26f9bce936" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:30.582682 2886 raft_consensus.cc:3037] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.582967 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6346d5ea09834bddb483da26f9bce936" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:30.583058 3019 raft_consensus.cc:3037] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.583566 2886 raft_consensus.cc:2454] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:30.583909 3219 leader_election.cc:302] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:30.584189 3870 raft_consensus.cc:2781] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:30.584336 3870 raft_consensus.cc:686] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:30.584457 3870 consensus_queue.cc:237] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.583802 3019 raft_consensus.cc:2454] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:30.586159 32730 catalog_manager.cc:5484] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:30.626178 3974 raft_consensus.cc:482] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:30.626297 3974 raft_consensus.cc:504] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.626780 3974 leader_election.cc:288] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:30.631493 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07610855449c4cd7a9ee4a99c477f73a" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:30.631666 3284 raft_consensus.cc:2454] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:30.632032 607 leader_election.cc:302] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:30.632206 3974 raft_consensus.cc:2781] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:30.632292 3974 raft_consensus.cc:482] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:30.632349 3974 raft_consensus.cc:3037] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.632438 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07610855449c4cd7a9ee4a99c477f73a" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:30.632581 3151 raft_consensus.cc:2454] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:30.633424 3974 raft_consensus.cc:504] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.633738 3974 leader_election.cc:288] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:30.634064 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07610855449c4cd7a9ee4a99c477f73a" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:30.634065 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07610855449c4cd7a9ee4a99c477f73a" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:30.634186 3151 raft_consensus.cc:3037] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.634186 3284 raft_consensus.cc:3037] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.635133 3284 raft_consensus.cc:2454] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:30.635133 3151 raft_consensus.cc:2454] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:30.635406 607 leader_election.cc:302] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:30.635556 3974 raft_consensus.cc:2781] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:30.635632 3974 raft_consensus.cc:686] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:30.635756 3974 consensus_queue.cc:237] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:30.637965 32730 catalog_manager.cc:5484] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:30.776875 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:30.781392 3284 raft_consensus.cc:1261] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.781478 3151 raft_consensus.cc:1261] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.781797 3859 consensus_queue.cc:1035] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.781941 3974 consensus_queue.cc:1035] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.784469 32730 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 07610855449c4cd7a9ee4a99c477f73a (table rebalancer_dynamic_table [id=9030555d09c0495b8113952059600b46]). Expected version 1 got 0
I20240502 14:00:30.784998 32733 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 07610855449c4cd7a9ee4a99c477f73a (table rebalancer_dynamic_table [id=9030555d09c0495b8113952059600b46]). Expected version 1 got 0
I20240502 14:00:30.786887 32733 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 07610855449c4cd7a9ee4a99c477f73a (table rebalancer_dynamic_table [id=9030555d09c0495b8113952059600b46]). Expected version 1 got 0
I20240502 14:00:30.786877 4074 tablet.cc:1722] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:30.787110 4076 tablet.cc:1722] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:30.787431 4076 tablet.cc:1620] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:30.787600 32730 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 07610855449c4cd7a9ee4a99c477f73a (table rebalancer_dynamic_table [id=9030555d09c0495b8113952059600b46]). Expected version 1 got 0
I20240502 14:00:30.787953 4074 tablet.cc:1620] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:30.788581 4077 tablet.cc:1722] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:30.788892 4077 tablet.cc:1620] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:30.788929 32730 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 07610855449c4cd7a9ee4a99c477f73a (table rebalancer_dynamic_table [id=9030555d09c0495b8113952059600b46]). Expected version 1 got 0
I20240502 14:00:30.791630 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=9030555d09c0495b8113952059600b46] alter complete (version 1)
I20240502 14:00:30.870642 3503 raft_consensus.cc:482] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:30.870769 3503 raft_consensus.cc:504] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:30.871217 3503 leader_election.cc:288] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:30.871665 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec443abbb4954bf3b28336e0b5bc0c4b" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:30.871793 3018 raft_consensus.cc:2454] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:30.872205 2819 leader_election.cc:302] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:30.872399 3503 raft_consensus.cc:2781] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:30.872465 3503 raft_consensus.cc:482] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:30.872511 3503 raft_consensus.cc:3037] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.873646 3503 raft_consensus.cc:504] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:30.874020 3503 leader_election.cc:288] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:30.874442 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec443abbb4954bf3b28336e0b5bc0c4b" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:30.874558 3018 raft_consensus.cc:3037] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.875576 3018 raft_consensus.cc:2454] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:30.876040 2819 leader_election.cc:302] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:30.876299 3503 raft_consensus.cc:2781] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:30.876376 3503 raft_consensus.cc:686] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:30.876509 3503 consensus_queue.cc:237] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:30.876902 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec443abbb4954bf3b28336e0b5bc0c4b" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:30.877018 405 raft_consensus.cc:2454] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:30.877781 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec443abbb4954bf3b28336e0b5bc0c4b" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:30.877880 409 raft_consensus.cc:3037] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.878826 409 raft_consensus.cc:2454] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:30.878847 32730 catalog_manager.cc:5484] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:30.892665 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:30.898265 4076 tablet.cc:1722] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:30.898363 4074 tablet.cc:1722] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:30.898603 4074 tablet.cc:1620] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:30.898603 4076 tablet.cc:1620] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:30.899688 4077 tablet.cc:1722] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:30.900012 4077 tablet.cc:1620] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:30.904700 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=9030555d09c0495b8113952059600b46] alter complete (version 2)
I20240502 14:00:30.978461 3870 consensus_queue.cc:1035] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.981904 3870 consensus_queue.cc:1035] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.006799 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:31.011750 4074 tablet.cc:1722] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:31.012332 4077 tablet.cc:1722] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:31.014400 4076 tablet.cc:1722] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:31.017988 3748 consensus_queue.cc:1035] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.021036 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=9030555d09c0495b8113952059600b46] alter complete (version 3)
I20240502 14:00:31.021414 4090 consensus_queue.cc:1035] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.124436 32730 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:31.124603 32730 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:31.126756 32730 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 07610855449c4cd7a9ee4a99c477f73a
I20240502 14:00:31.127339 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet 07610855449c4cd7a9ee4a99c477f73a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:31 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:31.127431 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet 07610855449c4cd7a9ee4a99c477f73a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:31 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:31.127439 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 07610855449c4cd7a9ee4a99c477f73a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:31 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:31.127718 4093 tablet_replica.cc:330] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:31.127848 4092 tablet_replica.cc:330] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:31.127727 4094 tablet_replica.cc:330] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:31.127944 4093 raft_consensus.cc:2229] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:31.128015 4092 raft_consensus.cc:2229] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:31.128072 4093 raft_consensus.cc:2258] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:31.128082 4094 raft_consensus.cc:2229] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:31.128145 4092 raft_consensus.cc:2258] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:31.128278 4094 raft_consensus.cc:2258] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:31.128671 4093 ts_tablet_manager.cc:1875] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:31.128801 4092 ts_tablet_manager.cc:1875] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:31.128857 4094 ts_tablet_manager.cc:1875] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:31.130566 4093 ts_tablet_manager.cc:1888] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:31.130641 4093 log.cc:1196] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/07610855449c4cd7a9ee4a99c477f73a
I20240502 14:00:31.130921 4093 ts_tablet_manager.cc:1909] T 07610855449c4cd7a9ee4a99c477f73a P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:31.131117 4092 ts_tablet_manager.cc:1888] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:31.131343 4092 log.cc:1196] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/07610855449c4cd7a9ee4a99c477f73a
I20240502 14:00:31.131381 4094 ts_tablet_manager.cc:1888] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:31.131467 4094 log.cc:1196] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/07610855449c4cd7a9ee4a99c477f73a
I20240502 14:00:31.131667 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 07610855449c4cd7a9ee4a99c477f73a (table rebalancer_dynamic_table_new_name [id=9030555d09c0495b8113952059600b46]) successfully deleted
I20240502 14:00:31.131732 4092 ts_tablet_manager.cc:1909] T 07610855449c4cd7a9ee4a99c477f73a P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:31.132068 4094 ts_tablet_manager.cc:1909] T 07610855449c4cd7a9ee4a99c477f73a P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:31.132704 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 07610855449c4cd7a9ee4a99c477f73a (table rebalancer_dynamic_table_new_name [id=9030555d09c0495b8113952059600b46]) successfully deleted
I20240502 14:00:31.132800 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 07610855449c4cd7a9ee4a99c477f73a (table rebalancer_dynamic_table_new_name [id=9030555d09c0495b8113952059600b46]) successfully deleted
I20240502 14:00:31.228650 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:31.233546 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 5e9c8504c47448f09f0f736bbc969b45 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=dd578a48ea364a1bafac37fa18a91847]), partition=
I20240502 14:00:31.233633 784 tablet_service.cc:1467] Processing CreateTablet for tablet 5e9c8504c47448f09f0f736bbc969b45 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=dd578a48ea364a1bafac37fa18a91847]), partition=
I20240502 14:00:31.233613 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 5e9c8504c47448f09f0f736bbc969b45 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=dd578a48ea364a1bafac37fa18a91847]), partition=
I20240502 14:00:31.233850 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e9c8504c47448f09f0f736bbc969b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.233850 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e9c8504c47448f09f0f736bbc969b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.233872 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e9c8504c47448f09f0f736bbc969b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.235783 4095 tablet_bootstrap.cc:492] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:31.236593 4096 tablet_bootstrap.cc:492] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:31.236863 4095 tablet_bootstrap.cc:654] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.237452 4096 tablet_bootstrap.cc:654] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.238313 4095 tablet_bootstrap.cc:492] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:31.238432 4095 ts_tablet_manager.cc:1367] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:31.238592 4096 tablet_bootstrap.cc:492] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:31.238678 4096 ts_tablet_manager.cc:1367] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:31.238937 4095 raft_consensus.cc:348] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:31.239106 4095 raft_consensus.cc:374] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.239075 4096 raft_consensus.cc:348] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:31.239151 4095 raft_consensus.cc:729] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.239212 4096 raft_consensus.cc:374] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.239287 4096 raft_consensus.cc:729] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.239280 4095 consensus_queue.cc:260] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:31.239436 4096 consensus_queue.cc:260] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:31.239596 4095 ts_tablet_manager.cc:1398] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:31.239770 4096 ts_tablet_manager.cc:1398] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:31.244235 4097 tablet_bootstrap.cc:492] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:31.245237 4097 tablet_bootstrap.cc:654] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.249440 4097 tablet_bootstrap.cc:492] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:31.249547 4097 ts_tablet_manager.cc:1367] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:31.250101 4097 raft_consensus.cc:348] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:31.250231 4097 raft_consensus.cc:374] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.250283 4097 raft_consensus.cc:729] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.250424 4097 consensus_queue.cc:260] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:31.250752 4097 ts_tablet_manager.cc:1398] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:31.254323 3503 raft_consensus.cc:482] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:31.254415 3503 raft_consensus.cc:504] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:31.254803 3503 leader_election.cc:288] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:31.255137 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e9c8504c47448f09f0f736bbc969b45" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:31.255264 804 raft_consensus.cc:2454] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:31.255640 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e9c8504c47448f09f0f736bbc969b45" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:31.255667 2822 leader_election.cc:302] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:31.255764 3283 raft_consensus.cc:2454] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:31.256325 3956 raft_consensus.cc:2781] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:31.256385 3956 raft_consensus.cc:482] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:31.256424 3956 raft_consensus.cc:3037] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.257380 3956 raft_consensus.cc:504] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:31.257645 3956 leader_election.cc:288] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:31.257946 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e9c8504c47448f09f0f736bbc969b45" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:31.257978 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e9c8504c47448f09f0f736bbc969b45" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:31.258062 804 raft_consensus.cc:3037] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.258077 3283 raft_consensus.cc:3037] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.258994 3283 raft_consensus.cc:2454] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:31.258993 804 raft_consensus.cc:2454] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:31.259284 2822 leader_election.cc:302] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:31.259471 3956 raft_consensus.cc:2781] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:31.259542 3956 raft_consensus.cc:686] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:31.259672 3956 consensus_queue.cc:237] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:31.261312 32730 catalog_manager.cc:5484] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:31.283190 3956 consensus_queue.cc:1035] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.287056 4103 consensus_queue.cc:1035] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.364334 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:31.368549 804 raft_consensus.cc:1261] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:31.368549 3283 raft_consensus.cc:1261] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:31.368995 3956 consensus_queue.cc:1035] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.369151 4103 consensus_queue.cc:1035] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.372017 32733 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 5e9c8504c47448f09f0f736bbc969b45 (table rebalancer_dynamic_table [id=dd578a48ea364a1bafac37fa18a91847]). Expected version 1 got 0
I20240502 14:00:31.372437 32730 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet 5e9c8504c47448f09f0f736bbc969b45 (table rebalancer_dynamic_table [id=dd578a48ea364a1bafac37fa18a91847]). Expected version 1 got 0
I20240502 14:00:31.373162 4108 tablet.cc:1722] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:31.373497 4108 tablet.cc:1620] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:31.373854 32733 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet 5e9c8504c47448f09f0f736bbc969b45 (table rebalancer_dynamic_table [id=dd578a48ea364a1bafac37fa18a91847]). Expected version 1 got 0
I20240502 14:00:31.373665 4109 tablet.cc:1722] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:31.374285 4109 tablet.cc:1620] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:31.374294 4077 tablet.cc:1722] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:31.374589 4077 tablet.cc:1620] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:31.375593 32733 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 5e9c8504c47448f09f0f736bbc969b45 (table rebalancer_dynamic_table [id=dd578a48ea364a1bafac37fa18a91847]). Expected version 1 got 0
I20240502 14:00:31.379837 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=dd578a48ea364a1bafac37fa18a91847] alter complete (version 1)
I20240502 14:00:31.391767 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:31.397785 2998 tablet_service.cc:1467] Processing CreateTablet for tablet c31fba4afb0045ca8af2f8a62721b892 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=de113df849174bb5a42c2527f02cac0d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.398123 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31fba4afb0045ca8af2f8a62721b892. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.398985 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 2f2c438f3a854f528ae8555cdb13721b (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=de113df849174bb5a42c2527f02cac0d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.399295 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f2c438f3a854f528ae8555cdb13721b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.399502 2866 tablet_service.cc:1467] Processing CreateTablet for tablet c31fba4afb0045ca8af2f8a62721b892 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=de113df849174bb5a42c2527f02cac0d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.399762 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31fba4afb0045ca8af2f8a62721b892. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.400069 2864 tablet_service.cc:1467] Processing CreateTablet for tablet db0bf9f6821045e1a259703fc3445226 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=de113df849174bb5a42c2527f02cac0d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.400280 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db0bf9f6821045e1a259703fc3445226. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.400547 522 tablet_service.cc:1467] Processing CreateTablet for tablet db0bf9f6821045e1a259703fc3445226 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=de113df849174bb5a42c2527f02cac0d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.400804 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db0bf9f6821045e1a259703fc3445226. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.402139 4097 tablet_bootstrap.cc:492] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:31.398984 3129 tablet_service.cc:1467] Processing CreateTablet for tablet db0bf9f6821045e1a259703fc3445226 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=de113df849174bb5a42c2527f02cac0d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.402349 3129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db0bf9f6821045e1a259703fc3445226. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.403055 4097 tablet_bootstrap.cc:654] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.403906 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 2f2c438f3a854f528ae8555cdb13721b (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=de113df849174bb5a42c2527f02cac0d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.404034 4112 tablet_bootstrap.cc:492] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:31.404167 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f2c438f3a854f528ae8555cdb13721b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.404587 653 tablet_service.cc:1467] Processing CreateTablet for tablet c31fba4afb0045ca8af2f8a62721b892 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=de113df849174bb5a42c2527f02cac0d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.404871 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31fba4afb0045ca8af2f8a62721b892. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.404942 4112 tablet_bootstrap.cc:654] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.405751 4111 tablet_bootstrap.cc:492] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:31.406174 4112 tablet_bootstrap.cc:492] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:31.406273 4112 ts_tablet_manager.cc:1367] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:31.406594 4111 tablet_bootstrap.cc:654] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.406730 4112 raft_consensus.cc:348] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:31.406867 4112 raft_consensus.cc:374] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.406940 4112 raft_consensus.cc:729] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.407110 4112 consensus_queue.cc:260] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:31.407545 4112 ts_tablet_manager.cc:1398] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:31.407780 4111 tablet_bootstrap.cc:492] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:31.407785 4116 tablet_bootstrap.cc:492] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:31.407958 4111 ts_tablet_manager.cc:1367] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:31.408447 4111 raft_consensus.cc:348] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:31.408619 4111 raft_consensus.cc:374] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.408702 4111 raft_consensus.cc:729] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.408792 4116 tablet_bootstrap.cc:654] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.408849 4111 consensus_queue.cc:260] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:31.409165 4111 ts_tablet_manager.cc:1398] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:31.409323 4111 tablet_bootstrap.cc:492] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:31.410029 4111 tablet_bootstrap.cc:654] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.410717 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 2f2c438f3a854f528ae8555cdb13721b (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=de113df849174bb5a42c2527f02cac0d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.411006 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f2c438f3a854f528ae8555cdb13721b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.412691 4110 tablet_bootstrap.cc:492] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:31.413260 4095 tablet_bootstrap.cc:492] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:31.413975 4111 tablet_bootstrap.cc:492] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:31.414074 4110 tablet_bootstrap.cc:654] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.414170 4111 ts_tablet_manager.cc:1367] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 14:00:31.414304 4095 tablet_bootstrap.cc:654] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.414839 4111 raft_consensus.cc:348] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:31.414966 4111 raft_consensus.cc:374] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.415027 4111 raft_consensus.cc:729] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.415166 4111 consensus_queue.cc:260] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:31.415259 4097 tablet_bootstrap.cc:492] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:31.415364 4097 ts_tablet_manager.cc:1367] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 14:00:31.415517 4111 ts_tablet_manager.cc:1398] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:31.415897 4097 raft_consensus.cc:348] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:31.416002 4097 raft_consensus.cc:374] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.416033 4097 raft_consensus.cc:729] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.416150 4097 consensus_queue.cc:260] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:31.416419 4095 tablet_bootstrap.cc:492] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:31.416437 4097 ts_tablet_manager.cc:1398] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:31.416489 4095 ts_tablet_manager.cc:1367] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:31.416616 4097 tablet_bootstrap.cc:492] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:31.416837 4095 raft_consensus.cc:348] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:31.416944 4095 raft_consensus.cc:374] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.416977 4095 raft_consensus.cc:729] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.417084 4095 consensus_queue.cc:260] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:31.417342 4095 ts_tablet_manager.cc:1398] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:31.417573 4097 tablet_bootstrap.cc:654] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.420269 4097 tablet_bootstrap.cc:492] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:31.420364 4097 ts_tablet_manager.cc:1367] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:31.420821 4097 raft_consensus.cc:348] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:31.420956 4097 raft_consensus.cc:374] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.420998 4097 raft_consensus.cc:729] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.421118 4097 consensus_queue.cc:260] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:31.421584 4097 ts_tablet_manager.cc:1398] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:31.421751 4110 tablet_bootstrap.cc:492] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:31.421849 4110 ts_tablet_manager.cc:1367] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:31.422328 4110 raft_consensus.cc:348] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:31.422456 4110 raft_consensus.cc:374] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.422503 4110 raft_consensus.cc:729] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.422647 4110 consensus_queue.cc:260] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:31.423023 4110 ts_tablet_manager.cc:1398] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:31.423213 4110 tablet_bootstrap.cc:492] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:31.424165 4110 tablet_bootstrap.cc:654] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.425457 4116 tablet_bootstrap.cc:492] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:31.425563 4116 ts_tablet_manager.cc:1367] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.002s
I20240502 14:00:31.425679 4110 tablet_bootstrap.cc:492] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:31.425753 4110 ts_tablet_manager.cc:1367] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:31.426018 4116 raft_consensus.cc:348] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:31.426107 4116 raft_consensus.cc:374] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.426128 4116 raft_consensus.cc:729] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.426149 4110 raft_consensus.cc:348] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:31.426247 4110 raft_consensus.cc:374] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.426256 4116 consensus_queue.cc:260] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:31.426292 4110 raft_consensus.cc:729] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.426402 4110 consensus_queue.cc:260] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:31.426527 4116 ts_tablet_manager.cc:1398] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:31.426702 4110 ts_tablet_manager.cc:1398] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:31.441164 4078 raft_consensus.cc:482] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:31.441278 4078 raft_consensus.cc:504] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:31.441704 4078 leader_election.cc:288] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:31.442096 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db0bf9f6821045e1a259703fc3445226" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:31.442251 3151 raft_consensus.cc:2454] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:31.442555 477 leader_election.cc:302] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:31.442727 4078 raft_consensus.cc:2781] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:31.442803 4078 raft_consensus.cc:482] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:31.442844 4078 raft_consensus.cc:3037] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.443768 4078 raft_consensus.cc:504] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:31.444144 4078 leader_election.cc:288] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:31.444470 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db0bf9f6821045e1a259703fc3445226" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:31.444590 3151 raft_consensus.cc:3037] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.445710 3151 raft_consensus.cc:2454] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:31.445991 477 leader_election.cc:302] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:31.446149 4078 raft_consensus.cc:2781] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:31.446303 4078 raft_consensus.cc:686] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:31.446471 4078 consensus_queue.cc:237] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:31.448146 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db0bf9f6821045e1a259703fc3445226" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:31.448225 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db0bf9f6821045e1a259703fc3445226" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:31.448292 2886 raft_consensus.cc:2454] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:31.448697 4084 raft_consensus.cc:482] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:31.448799 4084 raft_consensus.cc:504] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:31.448868 32733 catalog_manager.cc:5484] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:31.449177 4084 leader_election.cc:288] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:31.449505 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f2c438f3a854f528ae8555cdb13721b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:31.449613 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f2c438f3a854f528ae8555cdb13721b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:31.449656 3018 raft_consensus.cc:2454] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:31.449710 3151 raft_consensus.cc:2454] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:31.449985 3216 leader_election.cc:302] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:31.450428 4084 raft_consensus.cc:2781] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:31.450497 4084 raft_consensus.cc:482] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:31.450541 4084 raft_consensus.cc:3037] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.451355 4084 raft_consensus.cc:504] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:31.451630 4084 leader_election.cc:288] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:31.451927 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f2c438f3a854f528ae8555cdb13721b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:31.452011 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f2c438f3a854f528ae8555cdb13721b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:31.452049 3018 raft_consensus.cc:3037] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.452116 3151 raft_consensus.cc:3037] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.452991 3151 raft_consensus.cc:2454] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:31.452991 3018 raft_consensus.cc:2454] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:31.453271 3216 leader_election.cc:302] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:31.453457 4084 raft_consensus.cc:2781] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:31.453567 4084 raft_consensus.cc:686] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:31.453703 4084 consensus_queue.cc:237] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:31.455443 32733 catalog_manager.cc:5484] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:31.455853 3859 raft_consensus.cc:482] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:31.455952 3859 raft_consensus.cc:504] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:31.456441 3859 leader_election.cc:288] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:31.460623 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31fba4afb0045ca8af2f8a62721b892" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:31.460762 2884 raft_consensus.cc:2454] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:31.461115 609 leader_election.cc:302] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:31.461304 3859 raft_consensus.cc:2781] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:31.461365 3859 raft_consensus.cc:482] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:31.461393 3859 raft_consensus.cc:3037] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.461386 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31fba4afb0045ca8af2f8a62721b892" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:31.461516 3018 raft_consensus.cc:2454] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:31.462289 3859 raft_consensus.cc:504] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:31.462601 3859 leader_election.cc:288] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:31.462929 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31fba4afb0045ca8af2f8a62721b892" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:31.462954 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31fba4afb0045ca8af2f8a62721b892" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:31.463060 2884 raft_consensus.cc:3037] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.463060 3018 raft_consensus.cc:3037] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.464031 2884 raft_consensus.cc:2454] T c31fba4afb0045ca8af2f8a62721b892 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:31.464031 3018 raft_consensus.cc:2454] T c31fba4afb0045ca8af2f8a62721b892 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:31.464313 606 leader_election.cc:302] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:31.464541 3859 raft_consensus.cc:2781] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:31.464627 3859 raft_consensus.cc:686] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:31.464789 3859 consensus_queue.cc:237] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:31.466835 32733 catalog_manager.cc:5484] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:31.482748 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:31.487938 4108 tablet.cc:1722] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:31.488272 4108 tablet.cc:1620] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:31.488401 4077 tablet.cc:1722] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:31.488698 4077 tablet.cc:1620] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:31.490510 4109 tablet.cc:1722] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:31.490828 4109 tablet.cc:1620] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:31.494513 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=dd578a48ea364a1bafac37fa18a91847] alter complete (version 2)
I20240502 14:00:31.597750 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:31.603365 4108 tablet.cc:1722] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:31.603788 4109 tablet.cc:1722] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:31.604257 4077 tablet.cc:1722] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:31.608878 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=dd578a48ea364a1bafac37fa18a91847] alter complete (version 3)
I20240502 14:00:31.712288 32733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:31.712450 32733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:31.714689 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 5e9c8504c47448f09f0f736bbc969b45
I20240502 14:00:31.715297 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e9c8504c47448f09f0f736bbc969b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:31 UTC) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:31.715328 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e9c8504c47448f09f0f736bbc969b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:31 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:31.715569 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e9c8504c47448f09f0f736bbc969b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:31 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:31.715706 4134 tablet_replica.cc:330] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:31.715883 4134 raft_consensus.cc:2229] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:31.716029 4135 tablet_replica.cc:330] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:31.716204 4134 raft_consensus.cc:2258] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:31.716223 4135 raft_consensus.cc:2229] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:31.716367 4135 raft_consensus.cc:2258] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:31.716578 4133 tablet_replica.cc:330] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:31.716989 4133 raft_consensus.cc:2229] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:31.717044 4134 ts_tablet_manager.cc:1875] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:31.717155 4133 raft_consensus.cc:2258] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:31.717206 4135 ts_tablet_manager.cc:1875] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:31.718042 4133 ts_tablet_manager.cc:1875] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:31.719457 4134 ts_tablet_manager.cc:1888] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:31.719564 4135 ts_tablet_manager.cc:1888] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:31.719579 4134 log.cc:1196] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/5e9c8504c47448f09f0f736bbc969b45
I20240502 14:00:31.719669 4135 log.cc:1196] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/5e9c8504c47448f09f0f736bbc969b45
I20240502 14:00:31.719918 4134 ts_tablet_manager.cc:1909] T 5e9c8504c47448f09f0f736bbc969b45 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting consensus metadata
I20240502 14:00:31.720047 4135 ts_tablet_manager.cc:1909] T 5e9c8504c47448f09f0f736bbc969b45 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:31.720640 4133 ts_tablet_manager.cc:1888] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:31.720698 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 5e9c8504c47448f09f0f736bbc969b45 (table rebalancer_dynamic_table_new_name [id=dd578a48ea364a1bafac37fa18a91847]) successfully deleted
I20240502 14:00:31.720883 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 5e9c8504c47448f09f0f736bbc969b45 (table rebalancer_dynamic_table_new_name [id=dd578a48ea364a1bafac37fa18a91847]) successfully deleted
I20240502 14:00:31.720741 4133 log.cc:1196] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/5e9c8504c47448f09f0f736bbc969b45
I20240502 14:00:31.721246 4133 ts_tablet_manager.cc:1909] T 5e9c8504c47448f09f0f736bbc969b45 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:31.721974 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 5e9c8504c47448f09f0f736bbc969b45 (table rebalancer_dynamic_table_new_name [id=dd578a48ea364a1bafac37fa18a91847]) successfully deleted
I20240502 14:00:31.816926 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:31.822336 653 tablet_service.cc:1467] Processing CreateTablet for tablet 93e2c51784ed48a0befedb993771956e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=dd78f9731bda43a8b81abe5d224d2207]), partition=
I20240502 14:00:31.822399 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 93e2c51784ed48a0befedb993771956e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=dd78f9731bda43a8b81abe5d224d2207]), partition=
I20240502 14:00:31.822641 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93e2c51784ed48a0befedb993771956e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.822697 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93e2c51784ed48a0befedb993771956e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.824936 4116 tablet_bootstrap.cc:492] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:31.825168 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 93e2c51784ed48a0befedb993771956e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=dd78f9731bda43a8b81abe5d224d2207]), partition=
I20240502 14:00:31.825284 4095 tablet_bootstrap.cc:492] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:31.825416 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93e2c51784ed48a0befedb993771956e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.825798 4116 tablet_bootstrap.cc:654] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.826192 4095 tablet_bootstrap.cc:654] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.827404 4097 tablet_bootstrap.cc:492] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:31.828140 4116 tablet_bootstrap.cc:492] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:31.828205 4116 ts_tablet_manager.cc:1367] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:00:31.828392 4097 tablet_bootstrap.cc:654] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.828619 4116 raft_consensus.cc:348] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:31.828730 4116 raft_consensus.cc:374] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.828763 4116 raft_consensus.cc:729] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.828874 4116 consensus_queue.cc:260] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:31.828928 4095 tablet_bootstrap.cc:492] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:31.829010 4095 ts_tablet_manager.cc:1367] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:31.829186 4116 ts_tablet_manager.cc:1398] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:31.829494 4095 raft_consensus.cc:348] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:31.829603 4095 raft_consensus.cc:374] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.829651 4095 raft_consensus.cc:729] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.829782 4095 consensus_queue.cc:260] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:31.830076 4095 ts_tablet_manager.cc:1398] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:31.830222 4097 tablet_bootstrap.cc:492] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:31.830318 4097 ts_tablet_manager.cc:1367] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:31.830731 4097 raft_consensus.cc:348] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:31.830847 4097 raft_consensus.cc:374] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.830883 4097 raft_consensus.cc:729] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.831000 4097 consensus_queue.cc:260] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:31.831272 4097 ts_tablet_manager.cc:1398] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:31.840071 2884 raft_consensus.cc:3037] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.841324 4078 consensus_queue.cc:1035] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.844971 4125 consensus_queue.cc:1035] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.864485 4084 consensus_queue.cc:1035] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.867947 4063 consensus_queue.cc:1035] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.869534 3974 raft_consensus.cc:482] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:31.869634 3974 raft_consensus.cc:504] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:31.870012 3974 leader_election.cc:288] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:31.870419 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e2c51784ed48a0befedb993771956e" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:31.870589 2884 raft_consensus.cc:2454] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:31.870755 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e2c51784ed48a0befedb993771956e" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:31.870882 3283 raft_consensus.cc:2454] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:31.870903 609 leader_election.cc:302] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:31.871058 3859 raft_consensus.cc:2781] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:31.871110 3859 raft_consensus.cc:482] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:31.871146 3859 raft_consensus.cc:3037] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.872335 3859 raft_consensus.cc:504] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:31.872655 3859 leader_election.cc:288] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:31.872941 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e2c51784ed48a0befedb993771956e" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:31.872938 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e2c51784ed48a0befedb993771956e" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:31.873039 2884 raft_consensus.cc:3037] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.873044 3283 raft_consensus.cc:3037] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.873824 3283 raft_consensus.cc:2454] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:31.874042 2884 raft_consensus.cc:2454] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:31.874286 607 leader_election.cc:302] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:31.874462 3974 raft_consensus.cc:2781] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:31.874557 3974 raft_consensus.cc:686] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:31.874680 3974 consensus_queue.cc:237] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:31.876447 32730 catalog_manager.cc:5484] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:31.978466 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:31.981922 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:31.985612 784 tablet_service.cc:1467] Processing CreateTablet for tablet 72395d91ff48409693431748b3efeb6d (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7157a49edcbc4ab4a4b4cb35ef78edac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.985985 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72395d91ff48409693431748b3efeb6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.988612 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 61f063b5eb9f480f954826c56722a528 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7157a49edcbc4ab4a4b4cb35ef78edac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.988662 3129 tablet_service.cc:1467] Processing CreateTablet for tablet 183e2942e969418c8e993a426935a7eb (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7157a49edcbc4ab4a4b4cb35ef78edac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.988893 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61f063b5eb9f480f954826c56722a528. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.988951 3129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 183e2942e969418c8e993a426935a7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.991134 4151 tablet_bootstrap.cc:492] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:31.991134 4150 tablet_bootstrap.cc:492] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:31.992012 3262 tablet_service.cc:1467] Processing CreateTablet for tablet 61f063b5eb9f480f954826c56722a528 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7157a49edcbc4ab4a4b4cb35ef78edac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.992046 388 tablet_service.cc:1467] Processing CreateTablet for tablet 183e2942e969418c8e993a426935a7eb (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7157a49edcbc4ab4a4b4cb35ef78edac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.992044 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 72395d91ff48409693431748b3efeb6d (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7157a49edcbc4ab4a4b4cb35ef78edac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.992298 4151 tablet_bootstrap.cc:654] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.992306 3262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61f063b5eb9f480f954826c56722a528. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.992313 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 183e2942e969418c8e993a426935a7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.992655 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72395d91ff48409693431748b3efeb6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.992853 522 tablet_service.cc:1467] Processing CreateTablet for tablet 72395d91ff48409693431748b3efeb6d (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7157a49edcbc4ab4a4b4cb35ef78edac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.993108 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72395d91ff48409693431748b3efeb6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.993399 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 183e2942e969418c8e993a426935a7eb (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7157a49edcbc4ab4a4b4cb35ef78edac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.993645 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 183e2942e969418c8e993a426935a7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.994544 4095 tablet_bootstrap.cc:492] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:31.995505 4095 tablet_bootstrap.cc:654] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.995617 386 tablet_service.cc:1467] Processing CreateTablet for tablet 61f063b5eb9f480f954826c56722a528 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=7157a49edcbc4ab4a4b4cb35ef78edac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.995910 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61f063b5eb9f480f954826c56722a528. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.996910 4150 tablet_bootstrap.cc:654] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.997380 4155 tablet_bootstrap.cc:492] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:31.998570 4151 tablet_bootstrap.cc:492] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:31.998595 2884 raft_consensus.cc:1261] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:31.998715 4151 ts_tablet_manager.cc:1367] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240502 14:00:31.999361 4151 raft_consensus.cc:348] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:31.999481 4151 raft_consensus.cc:374] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.999514 4151 raft_consensus.cc:729] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.999688 4155 tablet_bootstrap.cc:654] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.999688 4151 consensus_queue.cc:260] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:32.000000 4151 ts_tablet_manager.cc:1398] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.000236 3974 consensus_queue.cc:1035] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.000928 3283 raft_consensus.cc:1261] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:32.001224 4155 tablet_bootstrap.cc:492] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:32.001312 4155 ts_tablet_manager.cc:1367] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:32.001343 3974 consensus_queue.cc:1035] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.001762 4155 raft_consensus.cc:348] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:32.001883 4155 raft_consensus.cc:374] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.001916 4155 raft_consensus.cc:729] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.002032 4155 consensus_queue.cc:260] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:32.002314 4155 ts_tablet_manager.cc:1398] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.002351 4157 tablet_bootstrap.cc:492] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:32.003778 4157 tablet_bootstrap.cc:654] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.004637 4108 tablet.cc:1722] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:32.004793 4152 tablet_bootstrap.cc:492] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:32.004889 32731 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet 93e2c51784ed48a0befedb993771956e (table rebalancer_dynamic_table [id=dd78f9731bda43a8b81abe5d224d2207]). Expected version 1 got 0
I20240502 14:00:32.004936 4108 tablet.cc:1620] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:32.005604 3859 consensus_queue.cc:1035] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.007393 4095 tablet_bootstrap.cc:492] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:32.007486 4095 ts_tablet_manager.cc:1367] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20240502 14:00:32.007526 4152 tablet_bootstrap.cc:654] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.008018 4095 raft_consensus.cc:348] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:32.008138 4095 raft_consensus.cc:374] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.008175 4095 raft_consensus.cc:729] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.008288 4095 consensus_queue.cc:260] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:32.008399 32731 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 93e2c51784ed48a0befedb993771956e (table rebalancer_dynamic_table [id=dd78f9731bda43a8b81abe5d224d2207]). Expected version 1 got 0
I20240502 14:00:32.008576 4095 ts_tablet_manager.cc:1398] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.008728 4095 tablet_bootstrap.cc:492] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:32.009100 32730 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 93e2c51784ed48a0befedb993771956e (table rebalancer_dynamic_table [id=dd78f9731bda43a8b81abe5d224d2207]). Expected version 1 got 0
I20240502 14:00:32.009650 4095 tablet_bootstrap.cc:654] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.010946 4150 tablet_bootstrap.cc:492] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:32.011036 4150 ts_tablet_manager.cc:1367] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 14:00:32.011546 4150 raft_consensus.cc:348] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:32.011649 4150 raft_consensus.cc:374] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.011672 4150 raft_consensus.cc:729] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.011775 4150 consensus_queue.cc:260] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:32.012068 4150 ts_tablet_manager.cc:1398] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.012876 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=dd78f9731bda43a8b81abe5d224d2207] alter complete (version 1)
I20240502 14:00:32.014083 4157 tablet_bootstrap.cc:492] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:32.014164 4157 ts_tablet_manager.cc:1367] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 14:00:32.014331 4152 tablet_bootstrap.cc:492] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:32.014410 4152 ts_tablet_manager.cc:1367] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:32.014629 4157 raft_consensus.cc:348] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:32.014729 4157 raft_consensus.cc:374] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.014758 4157 raft_consensus.cc:729] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.014894 4157 consensus_queue.cc:260] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:32.014880 4152 raft_consensus.cc:348] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:32.014977 4152 raft_consensus.cc:374] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.014999 4152 raft_consensus.cc:729] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.015120 4152 consensus_queue.cc:260] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:32.015159 4157 ts_tablet_manager.cc:1398] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.015305 4157 tablet_bootstrap.cc:492] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:32.015393 4152 ts_tablet_manager.cc:1398] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.016240 4157 tablet_bootstrap.cc:654] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.018108 4157 tablet_bootstrap.cc:492] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:32.018193 4157 ts_tablet_manager.cc:1367] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:32.018693 4157 raft_consensus.cc:348] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:32.019631 4077 tablet.cc:1722] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:32.019964 4077 tablet.cc:1620] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:32.022102 32726 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 93e2c51784ed48a0befedb993771956e (table rebalancer_dynamic_table [id=dd78f9731bda43a8b81abe5d224d2207]). Expected version 1 got 0
I20240502 14:00:32.022125 3859 consensus_queue.cc:1035] T c31fba4afb0045ca8af2f8a62721b892 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.024291 32731 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 93e2c51784ed48a0befedb993771956e (table rebalancer_dynamic_table [id=dd78f9731bda43a8b81abe5d224d2207]). Expected version 1 got 0
I20240502 14:00:32.018798 4157 raft_consensus.cc:374] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.026031 4157 raft_consensus.cc:729] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.026253 4157 consensus_queue.cc:260] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:32.027979 4157 ts_tablet_manager.cc:1398] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20240502 14:00:32.033624 4163 tablet.cc:1722] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:32.033975 4163 tablet.cc:1620] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:32.038197 4095 tablet_bootstrap.cc:492] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:32.038288 4095 ts_tablet_manager.cc:1367] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240502 14:00:32.038745 4095 raft_consensus.cc:348] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:32.038858 4095 raft_consensus.cc:374] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.038889 4095 raft_consensus.cc:729] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.039000 4095 consensus_queue.cc:260] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:32.039340 4095 ts_tablet_manager.cc:1398] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.039526 4095 tablet_bootstrap.cc:492] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:32.040514 4095 tablet_bootstrap.cc:654] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.041898 4095 tablet_bootstrap.cc:492] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:32.042131 4095 ts_tablet_manager.cc:1367] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:32.044306 4095 raft_consensus.cc:348] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:32.044485 4095 raft_consensus.cc:374] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.044556 4095 raft_consensus.cc:729] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.044756 4095 consensus_queue.cc:260] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:32.045209 4095 ts_tablet_manager.cc:1398] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:32.047998 4063 raft_consensus.cc:482] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.048084 4063 raft_consensus.cc:504] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:32.048444 4063 leader_election.cc:288] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:32.048899 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183e2942e969418c8e993a426935a7eb" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:32.049085 3151 raft_consensus.cc:2454] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:32.049738 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183e2942e969418c8e993a426935a7eb" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:32.049890 405 raft_consensus.cc:2454] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:32.050379 3219 leader_election.cc:302] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:32.050566 4063 raft_consensus.cc:2781] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.050626 4063 raft_consensus.cc:482] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.050666 4063 raft_consensus.cc:3037] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.051684 4063 raft_consensus.cc:504] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:32.052130 4063 leader_election.cc:288] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:32.052356 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183e2942e969418c8e993a426935a7eb" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:32.052462 405 raft_consensus.cc:3037] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.052908 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "183e2942e969418c8e993a426935a7eb" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:32.053035 3151 raft_consensus.cc:3037] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.053387 405 raft_consensus.cc:2454] T 183e2942e969418c8e993a426935a7eb P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:32.053787 3218 leader_election.cc:302] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:32.053931 4063 raft_consensus.cc:2781] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.054000 4063 raft_consensus.cc:686] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:32.054116 3151 raft_consensus.cc:2454] T 183e2942e969418c8e993a426935a7eb P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:32.054116 4063 consensus_queue.cc:237] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:32.056802 32726 catalog_manager.cc:5484] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:32.101295 4090 raft_consensus.cc:482] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.101442 4090 raft_consensus.cc:504] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:32.102063 4090 leader_election.cc:288] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:32.102636 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61f063b5eb9f480f954826c56722a528" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:32.102808 3018 raft_consensus.cc:2454] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:32.103103 341 leader_election.cc:302] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:32.103283 4090 raft_consensus.cc:2781] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.103351 4090 raft_consensus.cc:482] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.103403 4090 raft_consensus.cc:3037] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.104568 4090 raft_consensus.cc:504] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:32.104852 4090 leader_election.cc:288] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:32.105262 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61f063b5eb9f480f954826c56722a528" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:32.105381 3018 raft_consensus.cc:3037] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.106297 3018 raft_consensus.cc:2454] T 61f063b5eb9f480f954826c56722a528 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:32.106606 341 leader_election.cc:302] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:32.106776 4090 raft_consensus.cc:2781] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.106878 4090 raft_consensus.cc:686] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:32.107019 4090 consensus_queue.cc:237] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:32.107848 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61f063b5eb9f480f954826c56722a528" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:32.107985 3283 raft_consensus.cc:2454] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:32.108166 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61f063b5eb9f480f954826c56722a528" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:32.108242 3284 raft_consensus.cc:3037] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.109103 3284 raft_consensus.cc:2454] T 61f063b5eb9f480f954826c56722a528 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:32.109428 32726 catalog_manager.cc:5484] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:32.124886 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:32.129779 4163 tablet.cc:1722] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:32.130118 4163 tablet.cc:1620] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:32.130169 4108 tablet.cc:1722] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:32.130450 4108 tablet.cc:1620] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:32.130759 4077 tablet.cc:1722] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:32.131068 4077 tablet.cc:1620] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:32.135290 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=dd78f9731bda43a8b81abe5d224d2207] alter complete (version 2)
I20240502 14:00:32.152582 3769 raft_consensus.cc:482] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.152738 3769 raft_consensus.cc:504] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:32.153198 3769 leader_election.cc:288] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:32.153523 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72395d91ff48409693431748b3efeb6d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:32.153568 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72395d91ff48409693431748b3efeb6d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:32.153695 3284 raft_consensus.cc:2454] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:32.153717 541 raft_consensus.cc:2454] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:32.153947 739 leader_election.cc:302] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:32.154099 3769 raft_consensus.cc:2781] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.154156 3769 raft_consensus.cc:482] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.154192 3769 raft_consensus.cc:3037] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.155133 3769 raft_consensus.cc:504] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:32.155475 3769 leader_election.cc:288] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:32.155773 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72395d91ff48409693431748b3efeb6d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:32.155853 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72395d91ff48409693431748b3efeb6d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:32.155925 541 raft_consensus.cc:3037] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.155984 3284 raft_consensus.cc:3037] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.156733 541 raft_consensus.cc:2454] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:32.156754 3284 raft_consensus.cc:2454] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:32.156991 738 leader_election.cc:302] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:32.157148 3769 raft_consensus.cc:2781] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.157289 3769 raft_consensus.cc:686] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:32.157423 3769 consensus_queue.cc:237] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:32.159212 32726 catalog_manager.cc:5484] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:32.237376 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:32.248730 4163 tablet.cc:1722] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:32.248863 4077 tablet.cc:1722] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:32.249385 4108 tablet.cc:1722] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:32.255208 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=dd78f9731bda43a8b81abe5d224d2207] alter complete (version 3)
I20240502 14:00:32.357820 32730 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:32.357976 32730 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:32.360148 32730 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 93e2c51784ed48a0befedb993771956e
I20240502 14:00:32.360741 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 93e2c51784ed48a0befedb993771956e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:32 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:32.360977 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 93e2c51784ed48a0befedb993771956e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:32 UTC) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:32.361055 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet 93e2c51784ed48a0befedb993771956e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:32 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:32.361299 4178 tablet_replica.cc:330] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:32.361371 4179 tablet_replica.cc:330] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:32.361403 4177 tablet_replica.cc:330] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:32.361466 4178 raft_consensus.cc:2229] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:32.361603 4178 raft_consensus.cc:2258] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:32.361590 4177 raft_consensus.cc:2229] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:32.361615 4179 raft_consensus.cc:2229] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:32.361860 4177 raft_consensus.cc:2258] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:32.361956 4179 raft_consensus.cc:2258] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:32.362270 4178 ts_tablet_manager.cc:1875] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:32.362522 4179 ts_tablet_manager.cc:1875] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:32.362584 4177 ts_tablet_manager.cc:1875] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:32.364920 4179 ts_tablet_manager.cc:1888] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:32.365038 4179 log.cc:1196] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/93e2c51784ed48a0befedb993771956e
I20240502 14:00:32.365088 4177 ts_tablet_manager.cc:1888] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:32.365185 4177 log.cc:1196] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/93e2c51784ed48a0befedb993771956e
I20240502 14:00:32.365347 4179 ts_tablet_manager.cc:1909] T 93e2c51784ed48a0befedb993771956e P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:32.365360 4178 ts_tablet_manager.cc:1888] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:32.365427 4178 log.cc:1196] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/93e2c51784ed48a0befedb993771956e
I20240502 14:00:32.365576 4177 ts_tablet_manager.cc:1909] T 93e2c51784ed48a0befedb993771956e P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:32.365705 4178 ts_tablet_manager.cc:1909] T 93e2c51784ed48a0befedb993771956e P 4bcd7712aa844d98977a7d7e605c31f5: Deleting consensus metadata
I20240502 14:00:32.366240 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 93e2c51784ed48a0befedb993771956e (table rebalancer_dynamic_table_new_name [id=dd78f9731bda43a8b81abe5d224d2207]) successfully deleted
I20240502 14:00:32.366240 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 93e2c51784ed48a0befedb993771956e (table rebalancer_dynamic_table_new_name [id=dd78f9731bda43a8b81abe5d224d2207]) successfully deleted
I20240502 14:00:32.366829 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 93e2c51784ed48a0befedb993771956e (table rebalancer_dynamic_table_new_name [id=dd78f9731bda43a8b81abe5d224d2207]) successfully deleted
I20240502 14:00:32.461928 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:32.467520 3129 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd6138d72eb49c0a69119fdf6974deb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=49435c1c1e6f43ba9a2efa76628e7c86]), partition=
I20240502 14:00:32.467540 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd6138d72eb49c0a69119fdf6974deb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=49435c1c1e6f43ba9a2efa76628e7c86]), partition=
I20240502 14:00:32.467808 3129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd6138d72eb49c0a69119fdf6974deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.467859 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd6138d72eb49c0a69119fdf6974deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.467875 653 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd6138d72eb49c0a69119fdf6974deb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=49435c1c1e6f43ba9a2efa76628e7c86]), partition=
I20240502 14:00:32.468086 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd6138d72eb49c0a69119fdf6974deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.469954 4181 tablet_bootstrap.cc:492] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:32.470582 4182 tablet_bootstrap.cc:492] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:32.470868 4181 tablet_bootstrap.cc:654] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.471432 4182 tablet_bootstrap.cc:654] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.472048 4181 tablet_bootstrap.cc:492] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:32.472348 4181 ts_tablet_manager.cc:1367] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:32.472429 4151 tablet_bootstrap.cc:492] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:32.472893 4181 raft_consensus.cc:348] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:32.473012 4181 raft_consensus.cc:374] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.473042 4181 raft_consensus.cc:729] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.473289 4151 tablet_bootstrap.cc:654] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.473632 4181 consensus_queue.cc:260] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:32.474088 4181 ts_tablet_manager.cc:1398] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:32.475021 4182 tablet_bootstrap.cc:492] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:32.475121 4182 ts_tablet_manager.cc:1367] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:32.475666 4182 raft_consensus.cc:348] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:32.475793 4182 raft_consensus.cc:374] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.475859 4182 raft_consensus.cc:729] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.476012 4182 consensus_queue.cc:260] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:32.476325 4182 ts_tablet_manager.cc:1398] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.477383 4151 tablet_bootstrap.cc:492] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:32.477481 4151 ts_tablet_manager.cc:1367] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:32.477996 4151 raft_consensus.cc:348] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:32.478120 4151 raft_consensus.cc:374] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.478153 4151 raft_consensus.cc:729] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.478282 4151 consensus_queue.cc:260] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:32.478585 4151 ts_tablet_manager.cc:1398] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.484105 4180 consensus_queue.cc:1035] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.489655 4063 consensus_queue.cc:1035] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.512042 4162 raft_consensus.cc:482] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.512166 4162 raft_consensus.cc:504] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:32.512589 4162 leader_election.cc:288] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:32.512950 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd6138d72eb49c0a69119fdf6974deb" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:32.513094 3152 raft_consensus.cc:2454] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:32.513038 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd6138d72eb49c0a69119fdf6974deb" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:32.513216 2886 raft_consensus.cc:2454] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:32.513417 609 leader_election.cc:302] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:32.513633 4162 raft_consensus.cc:2781] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.513693 4162 raft_consensus.cc:482] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.513743 4162 raft_consensus.cc:3037] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.514987 4162 raft_consensus.cc:504] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:32.515326 4162 leader_election.cc:288] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:32.515631 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd6138d72eb49c0a69119fdf6974deb" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:32.515671 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd6138d72eb49c0a69119fdf6974deb" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:32.515774 2886 raft_consensus.cc:3037] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.515774 3152 raft_consensus.cc:3037] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.516742 2886 raft_consensus.cc:2454] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:32.516742 3152 raft_consensus.cc:2454] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:32.517040 609 leader_election.cc:302] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:32.517189 4162 raft_consensus.cc:2781] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.517280 4162 raft_consensus.cc:686] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:32.517442 4162 consensus_queue.cc:237] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:32.518844 4132 consensus_queue.cc:1035] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.519706 32730 catalog_manager.cc:5484] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:32.522718 4132 consensus_queue.cc:1035] T 61f063b5eb9f480f954826c56722a528 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.633852 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:32.638111 3152 raft_consensus.cc:1261] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:32.638124 2886 raft_consensus.cc:1261] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:32.638500 4168 consensus_queue.cc:1035] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.638654 3859 consensus_queue.cc:1035] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.641726 32731 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 4fd6138d72eb49c0a69119fdf6974deb (table rebalancer_dynamic_table [id=49435c1c1e6f43ba9a2efa76628e7c86]). Expected version 1 got 0
I20240502 14:00:32.641887 4163 tablet.cc:1722] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:32.642084 32730 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet 4fd6138d72eb49c0a69119fdf6974deb (table rebalancer_dynamic_table [id=49435c1c1e6f43ba9a2efa76628e7c86]). Expected version 1 got 0
I20240502 14:00:32.642199 4163 tablet.cc:1620] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:32.643127 4108 tablet.cc:1722] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:32.643446 4108 tablet.cc:1620] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:32.644481 32730 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 4fd6138d72eb49c0a69119fdf6974deb (table rebalancer_dynamic_table [id=49435c1c1e6f43ba9a2efa76628e7c86]). Expected version 1 got 0
I20240502 14:00:32.645403 4197 tablet.cc:1722] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:32.645728 4197 tablet.cc:1620] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:32.646706 32730 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 4fd6138d72eb49c0a69119fdf6974deb (table rebalancer_dynamic_table [id=49435c1c1e6f43ba9a2efa76628e7c86]). Expected version 1 got 0
I20240502 14:00:32.649381 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=49435c1c1e6f43ba9a2efa76628e7c86] alter complete (version 1)
I20240502 14:00:32.654069 3769 consensus_queue.cc:1035] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.657541 3769 consensus_queue.cc:1035] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.669551 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:32.675544 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 00d3e2b37ce743bf90e211f5385efe90 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.675590 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 00d3e2b37ce743bf90e211f5385efe90 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.675918 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00d3e2b37ce743bf90e211f5385efe90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.675933 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00d3e2b37ce743bf90e211f5385efe90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.676357 786 tablet_service.cc:1467] Processing CreateTablet for tablet ad4e1b6dea1e4beeb4b82852ded83c5f (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.676617 786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad4e1b6dea1e4beeb4b82852ded83c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.676641 2998 tablet_service.cc:1467] Processing CreateTablet for tablet be6290bc7df6465f8a43aae431b34f8b (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.676903 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be6290bc7df6465f8a43aae431b34f8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.678385 522 tablet_service.cc:1467] Processing CreateTablet for tablet ad4e1b6dea1e4beeb4b82852ded83c5f (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.678406 784 tablet_service.cc:1467] Processing CreateTablet for tablet 00d3e2b37ce743bf90e211f5385efe90 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.678637 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00d3e2b37ce743bf90e211f5385efe90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.678649 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad4e1b6dea1e4beeb4b82852ded83c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.679158 4205 tablet_bootstrap.cc:492] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:32.680224 4205 tablet_bootstrap.cc:654] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.680357 4203 tablet_bootstrap.cc:492] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:32.681098 2866 tablet_service.cc:1467] Processing CreateTablet for tablet ad4e1b6dea1e4beeb4b82852ded83c5f (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.681203 4181 tablet_bootstrap.cc:492] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:32.681330 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad4e1b6dea1e4beeb4b82852ded83c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.681900 4207 tablet_bootstrap.cc:492] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:32.681952 4203 tablet_bootstrap.cc:654] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.682291 4181 tablet_bootstrap.cc:654] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.682636 3264 tablet_service.cc:1467] Processing CreateTablet for tablet be6290bc7df6465f8a43aae431b34f8b (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.682873 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be6290bc7df6465f8a43aae431b34f8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.682898 4207 tablet_bootstrap.cc:654] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.684036 4205 tablet_bootstrap.cc:492] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:32.684139 4205 ts_tablet_manager.cc:1367] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:32.684628 4205 raft_consensus.cc:348] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.684749 4205 raft_consensus.cc:374] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.684787 4205 raft_consensus.cc:729] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.684888 4205 consensus_queue.cc:260] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.681098 2864 tablet_service.cc:1467] Processing CreateTablet for tablet be6290bc7df6465f8a43aae431b34f8b (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.685266 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be6290bc7df6465f8a43aae431b34f8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.685743 4204 tablet_bootstrap.cc:492] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:32.686698 4204 tablet_bootstrap.cc:654] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.687466 4207 tablet_bootstrap.cc:492] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:32.687613 4207 ts_tablet_manager.cc:1367] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:32.687780 4181 tablet_bootstrap.cc:492] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:32.687887 4181 ts_tablet_manager.cc:1367] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:32.688266 4207 raft_consensus.cc:348] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.688377 4207 raft_consensus.cc:374] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.688355 4181 raft_consensus.cc:348] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:32.688411 4207 raft_consensus.cc:729] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.688455 4181 raft_consensus.cc:374] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.688490 4181 raft_consensus.cc:729] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.688542 4207 consensus_queue.cc:260] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.688611 4181 consensus_queue.cc:260] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:32.688813 4207 ts_tablet_manager.cc:1398] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:32.688889 4181 ts_tablet_manager.cc:1398] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:32.689033 4205 ts_tablet_manager.cc:1398] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:32.689044 4181 tablet_bootstrap.cc:492] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:32.689900 4181 tablet_bootstrap.cc:654] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.689975 4203 tablet_bootstrap.cc:492] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:32.690064 4203 ts_tablet_manager.cc:1367] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:32.690551 4203 raft_consensus.cc:348] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:32.690671 4203 raft_consensus.cc:374] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.690702 4203 raft_consensus.cc:729] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.690826 4203 consensus_queue.cc:260] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:32.691125 4203 ts_tablet_manager.cc:1398] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.691296 4203 tablet_bootstrap.cc:492] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:32.691382 4181 tablet_bootstrap.cc:492] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:32.691452 4181 ts_tablet_manager.cc:1367] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:32.691910 4181 raft_consensus.cc:348] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.692003 4181 raft_consensus.cc:374] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.692044 4181 raft_consensus.cc:729] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.692148 4181 consensus_queue.cc:260] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.692200 4204 tablet_bootstrap.cc:492] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:32.692288 4203 tablet_bootstrap.cc:654] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.692421 4181 ts_tablet_manager.cc:1398] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.692405 4204 ts_tablet_manager.cc:1367] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:32.692575 4181 tablet_bootstrap.cc:492] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:32.693028 4204 raft_consensus.cc:348] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.693131 4204 raft_consensus.cc:374] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.693166 4204 raft_consensus.cc:729] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.693300 4204 consensus_queue.cc:260] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.693397 4181 tablet_bootstrap.cc:654] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.694288 4204 ts_tablet_manager.cc:1398] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:32.694456 4204 tablet_bootstrap.cc:492] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:32.694633 4181 tablet_bootstrap.cc:492] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:32.694700 4181 ts_tablet_manager.cc:1367] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:32.695149 4181 raft_consensus.cc:348] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.695240 4181 raft_consensus.cc:374] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.695279 4181 raft_consensus.cc:729] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.695372 4204 tablet_bootstrap.cc:654] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.695413 4181 consensus_queue.cc:260] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.696370 4181 ts_tablet_manager.cc:1398] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:32.698587 4204 tablet_bootstrap.cc:492] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:32.698678 4204 ts_tablet_manager.cc:1367] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:32.699165 4204 raft_consensus.cc:348] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:32.699316 4204 raft_consensus.cc:374] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.699380 4204 raft_consensus.cc:729] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.699529 4204 consensus_queue.cc:260] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:32.700145 4203 tablet_bootstrap.cc:492] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:32.700241 4203 ts_tablet_manager.cc:1367] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 14:00:32.700456 4204 ts_tablet_manager.cc:1398] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:32.704092 4203 raft_consensus.cc:348] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.704233 4203 raft_consensus.cc:374] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.704262 4203 raft_consensus.cc:729] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.704380 4203 consensus_queue.cc:260] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.704736 4203 ts_tablet_manager.cc:1398] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:32.713260 4202 raft_consensus.cc:482] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.713359 4202 raft_consensus.cc:504] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.713783 4202 leader_election.cc:288] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:32.714262 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4e1b6dea1e4beeb4b82852ded83c5f" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:32.714426 805 raft_consensus.cc:2454] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:32.714476 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4e1b6dea1e4beeb4b82852ded83c5f" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:32.714618 2886 raft_consensus.cc:2454] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:32.714753 477 leader_election.cc:302] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:32.714982 4202 raft_consensus.cc:2781] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.715035 4202 raft_consensus.cc:482] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.715061 4202 raft_consensus.cc:3037] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.715962 4202 raft_consensus.cc:504] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.716266 4202 leader_election.cc:288] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:32.716646 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4e1b6dea1e4beeb4b82852ded83c5f" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:32.716682 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4e1b6dea1e4beeb4b82852ded83c5f" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:32.716758 2886 raft_consensus.cc:3037] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.716848 805 raft_consensus.cc:3037] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.717715 2886 raft_consensus.cc:2454] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:32.717823 805 raft_consensus.cc:2454] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:32.718014 477 leader_election.cc:302] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:32.718297 4202 raft_consensus.cc:2781] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.718374 4202 raft_consensus.cc:686] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:32.718510 4202 consensus_queue.cc:237] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.720592 32726 catalog_manager.cc:5484] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:32.752061 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:32.757385 4163 tablet.cc:1722] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:32.757733 4163 tablet.cc:1620] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:32.757920 4197 tablet.cc:1722] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:32.757916 4108 tablet.cc:1722] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:32.758205 4197 tablet.cc:1620] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:32.758229 4108 tablet.cc:1620] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:32.763378 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=49435c1c1e6f43ba9a2efa76628e7c86] alter complete (version 2)
I20240502 14:00:32.799054 3956 raft_consensus.cc:482] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.799181 3956 raft_consensus.cc:504] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:32.799615 3956 leader_election.cc:288] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:32.800032 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d3e2b37ce743bf90e211f5385efe90" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:32.800091 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d3e2b37ce743bf90e211f5385efe90" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:32.800189 3284 raft_consensus.cc:2454] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:32.800220 805 raft_consensus.cc:2454] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:32.800539 2820 leader_election.cc:302] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:32.800756 3956 raft_consensus.cc:2781] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.800817 3956 raft_consensus.cc:482] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.800849 3956 raft_consensus.cc:3037] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.801888 3956 raft_consensus.cc:504] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:32.802239 3956 leader_election.cc:288] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:32.802563 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d3e2b37ce743bf90e211f5385efe90" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:32.802551 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d3e2b37ce743bf90e211f5385efe90" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:32.802673 3284 raft_consensus.cc:3037] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.802680 805 raft_consensus.cc:3037] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.803478 805 raft_consensus.cc:2454] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:32.803601 3284 raft_consensus.cc:2454] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:32.803781 2822 leader_election.cc:302] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:32.804189 3956 raft_consensus.cc:2781] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.804292 3956 raft_consensus.cc:686] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:32.804431 3956 consensus_queue.cc:237] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:32.806408 32731 catalog_manager.cc:5484] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:32.808472 4084 raft_consensus.cc:482] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.808570 4084 raft_consensus.cc:504] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.808939 4084 leader_election.cc:288] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:32.809290 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6290bc7df6465f8a43aae431b34f8b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:32.809437 3018 raft_consensus.cc:2454] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:32.809659 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6290bc7df6465f8a43aae431b34f8b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:32.809713 3216 leader_election.cc:302] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:32.809875 4084 raft_consensus.cc:2781] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.809932 4084 raft_consensus.cc:482] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.809988 4084 raft_consensus.cc:3037] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.809973 2886 raft_consensus.cc:2454] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:32.810971 4084 raft_consensus.cc:504] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.811278 4084 leader_election.cc:288] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:32.811692 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6290bc7df6465f8a43aae431b34f8b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:32.811815 3018 raft_consensus.cc:3037] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.811949 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6290bc7df6465f8a43aae431b34f8b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:32.812068 2886 raft_consensus.cc:3037] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.812829 3018 raft_consensus.cc:2454] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:32.812829 2886 raft_consensus.cc:2454] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:32.813148 3216 leader_election.cc:302] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:32.813304 4084 raft_consensus.cc:2781] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.813424 4084 raft_consensus.cc:686] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:32.813577 4084 consensus_queue.cc:237] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:32.815713 32731 catalog_manager.cc:5484] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:32.865355 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:32.870522 4163 tablet.cc:1722] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:32.870962 4108 tablet.cc:1722] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:32.871126 4197 tablet.cc:1722] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:32.875581 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=49435c1c1e6f43ba9a2efa76628e7c86] alter complete (version 3)
I20240502 14:00:32.977187 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:32.977396 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:32.979962 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 4fd6138d72eb49c0a69119fdf6974deb
I20240502 14:00:32.980520 3129 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fd6138d72eb49c0a69119fdf6974deb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:32 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:32.980638 2864 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fd6138d72eb49c0a69119fdf6974deb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:32 UTC) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:32.980803 4220 tablet_replica.cc:330] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:32.980813 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fd6138d72eb49c0a69119fdf6974deb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:32 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:32.980949 4221 tablet_replica.cc:330] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:32.980990 4220 raft_consensus.cc:2229] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:32.981067 4221 raft_consensus.cc:2229] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:32.981148 4220 raft_consensus.cc:2258] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:32.981194 4221 raft_consensus.cc:2258] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:32.981470 4222 tablet_replica.cc:330] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:32.981626 4222 raft_consensus.cc:2229] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:32.981794 4220 ts_tablet_manager.cc:1875] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:32.981832 4222 raft_consensus.cc:2258] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:32.981901 4221 ts_tablet_manager.cc:1875] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:32.982368 4222 ts_tablet_manager.cc:1875] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:32.983964 4220 ts_tablet_manager.cc:1888] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:32.984035 4220 log.cc:1196] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/4fd6138d72eb49c0a69119fdf6974deb
I20240502 14:00:32.984265 4221 ts_tablet_manager.cc:1888] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:32.984354 4221 log.cc:1196] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/4fd6138d72eb49c0a69119fdf6974deb
I20240502 14:00:32.984406 4220 ts_tablet_manager.cc:1909] T 4fd6138d72eb49c0a69119fdf6974deb P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:32.984709 4222 ts_tablet_manager.cc:1888] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:32.984740 4221 ts_tablet_manager.cc:1909] T 4fd6138d72eb49c0a69119fdf6974deb P 4bcd7712aa844d98977a7d7e605c31f5: Deleting consensus metadata
I20240502 14:00:32.984788 4222 log.cc:1196] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/4fd6138d72eb49c0a69119fdf6974deb
I20240502 14:00:32.985133 4222 ts_tablet_manager.cc:1909] T 4fd6138d72eb49c0a69119fdf6974deb P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:32.985119 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 4fd6138d72eb49c0a69119fdf6974deb (table rebalancer_dynamic_table_new_name [id=49435c1c1e6f43ba9a2efa76628e7c86]) successfully deleted
I20240502 14:00:32.985445 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 4fd6138d72eb49c0a69119fdf6974deb (table rebalancer_dynamic_table_new_name [id=49435c1c1e6f43ba9a2efa76628e7c86]) successfully deleted
I20240502 14:00:32.985699 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 4fd6138d72eb49c0a69119fdf6974deb (table rebalancer_dynamic_table_new_name [id=49435c1c1e6f43ba9a2efa76628e7c86]) successfully deleted
I20240502 14:00:33.082105 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:33.087291 3264 tablet_service.cc:1467] Processing CreateTablet for tablet b89e76ea92f847ca89e9fd2470455832 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b3c7d50540294e79a621956174314d15]), partition=
I20240502 14:00:33.087458 3129 tablet_service.cc:1467] Processing CreateTablet for tablet b89e76ea92f847ca89e9fd2470455832 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b3c7d50540294e79a621956174314d15]), partition=
I20240502 14:00:33.087550 386 tablet_service.cc:1467] Processing CreateTablet for tablet b89e76ea92f847ca89e9fd2470455832 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b3c7d50540294e79a621956174314d15]), partition=
I20240502 14:00:33.087610 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b89e76ea92f847ca89e9fd2470455832. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.087752 3129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b89e76ea92f847ca89e9fd2470455832. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.087786 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b89e76ea92f847ca89e9fd2470455832. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.089573 4224 tablet_bootstrap.cc:492] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:33.089911 4203 tablet_bootstrap.cc:492] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:33.090693 4225 tablet_bootstrap.cc:492] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:33.090845 4203 tablet_bootstrap.cc:654] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.090934 4224 tablet_bootstrap.cc:654] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.091719 4225 tablet_bootstrap.cc:654] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.091944 4203 tablet_bootstrap.cc:492] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:33.092053 4203 ts_tablet_manager.cc:1367] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 14:00:33.092514 4203 raft_consensus.cc:348] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:33.092633 4203 raft_consensus.cc:374] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.092666 4203 raft_consensus.cc:729] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.092798 4203 consensus_queue.cc:260] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:33.092851 4225 tablet_bootstrap.cc:492] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:33.092937 4225 ts_tablet_manager.cc:1367] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:33.093096 4224 tablet_bootstrap.cc:492] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:33.093109 4203 ts_tablet_manager.cc:1398] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:33.093168 4224 ts_tablet_manager.cc:1367] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:33.093385 4225 raft_consensus.cc:348] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:33.093499 4225 raft_consensus.cc:374] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.093538 4225 raft_consensus.cc:729] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.093606 4224 raft_consensus.cc:348] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:33.093663 4225 consensus_queue.cc:260] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:33.093713 4224 raft_consensus.cc:374] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.093745 4224 raft_consensus.cc:729] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.093874 4224 consensus_queue.cc:260] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:33.093950 4225 ts_tablet_manager.cc:1398] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:33.094201 4224 ts_tablet_manager.cc:1398] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:33.099812 4202 consensus_queue.cc:1035] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.103569 4202 consensus_queue.cc:1035] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.202034 4063 consensus_queue.cc:1035] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.207084 4180 consensus_queue.cc:1035] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.215093 3956 consensus_queue.cc:1035] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.219679 3956 consensus_queue.cc:1035] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.282780 4194 raft_consensus.cc:482] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:33.282923 4194 raft_consensus.cc:504] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:33.283330 4194 leader_election.cc:288] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:33.283738 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b89e76ea92f847ca89e9fd2470455832" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:33.283737 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b89e76ea92f847ca89e9fd2470455832" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:33.283926 405 raft_consensus.cc:2454] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:33.283941 3284 raft_consensus.cc:2454] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:33.284250 3086 leader_election.cc:302] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:33.284449 4194 raft_consensus.cc:2781] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:33.284526 4194 raft_consensus.cc:482] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:33.284575 4194 raft_consensus.cc:3037] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.285537 4194 raft_consensus.cc:504] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:33.285893 4194 leader_election.cc:288] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:33.286214 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b89e76ea92f847ca89e9fd2470455832" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:33.286235 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b89e76ea92f847ca89e9fd2470455832" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:33.286337 3284 raft_consensus.cc:3037] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.286334 405 raft_consensus.cc:3037] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.287087 3284 raft_consensus.cc:2454] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:33.287094 405 raft_consensus.cc:2454] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:33.287364 3085 leader_election.cc:302] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:33.287536 4194 raft_consensus.cc:2781] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:33.287645 4194 raft_consensus.cc:686] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:33.287792 4194 consensus_queue.cc:237] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:33.289770 32732 catalog_manager.cc:5484] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:33.341331 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:33.347492 2864 tablet_service.cc:1467] Processing CreateTablet for tablet 3bcaf05a01194e919efca9cc08d61c8f (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=8bbaf40cee36492e87629b1979f62041]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.347867 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bcaf05a01194e919efca9cc08d61c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.348609 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 3bcaf05a01194e919efca9cc08d61c8f (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=8bbaf40cee36492e87629b1979f62041]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.348781 3129 tablet_service.cc:1467] Processing CreateTablet for tablet 3bcaf05a01194e919efca9cc08d61c8f (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=8bbaf40cee36492e87629b1979f62041]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.348891 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bcaf05a01194e919efca9cc08d61c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.348801 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 8cf2d71983204c3c96d112cab1004478 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=8bbaf40cee36492e87629b1979f62041]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.349100 3129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bcaf05a01194e919efca9cc08d61c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.349217 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cf2d71983204c3c96d112cab1004478. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.350438 653 tablet_service.cc:1467] Processing CreateTablet for tablet 8cf2d71983204c3c96d112cab1004478 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=8bbaf40cee36492e87629b1979f62041]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.350711 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cf2d71983204c3c96d112cab1004478. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.351351 3132 tablet_service.cc:1467] Processing CreateTablet for tablet db51059f37184061914c56e322535314 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=8bbaf40cee36492e87629b1979f62041]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.351583 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db51059f37184061914c56e322535314. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.352025 4240 tablet_bootstrap.cc:492] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:33.353040 4241 tablet_bootstrap.cc:492] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:33.352874 4240 tablet_bootstrap.cc:654] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.353626 2866 tablet_service.cc:1467] Processing CreateTablet for tablet db51059f37184061914c56e322535314 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=8bbaf40cee36492e87629b1979f62041]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.353726 2864 tablet_service.cc:1467] Processing CreateTablet for tablet 8cf2d71983204c3c96d112cab1004478 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=8bbaf40cee36492e87629b1979f62041]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.353783 522 tablet_service.cc:1467] Processing CreateTablet for tablet db51059f37184061914c56e322535314 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=8bbaf40cee36492e87629b1979f62041]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.353874 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db51059f37184061914c56e322535314. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.353969 4241 tablet_bootstrap.cc:654] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.354053 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db51059f37184061914c56e322535314. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.354178 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cf2d71983204c3c96d112cab1004478. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.355947 4224 tablet_bootstrap.cc:492] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:33.356343 4239 tablet_bootstrap.cc:492] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:33.357121 4224 tablet_bootstrap.cc:654] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.357352 4240 tablet_bootstrap.cc:492] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:33.357426 4240 ts_tablet_manager.cc:1367] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:33.357538 4239 tablet_bootstrap.cc:654] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.358008 4240 raft_consensus.cc:348] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:33.358121 4240 raft_consensus.cc:374] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.358152 4240 raft_consensus.cc:729] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.358259 4240 consensus_queue.cc:260] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:33.358332 4224 tablet_bootstrap.cc:492] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:33.358402 4224 ts_tablet_manager.cc:1367] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:33.358471 4240 ts_tablet_manager.cc:1398] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:33.358822 4224 raft_consensus.cc:348] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:33.358925 4224 raft_consensus.cc:374] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.358965 4224 raft_consensus.cc:729] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.359081 4224 consensus_queue.cc:260] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:33.359364 4224 ts_tablet_manager.cc:1398] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:33.359514 4224 tablet_bootstrap.cc:492] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:33.360358 4224 tablet_bootstrap.cc:654] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.361873 4247 tablet_bootstrap.cc:492] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:33.361941 4224 tablet_bootstrap.cc:492] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:33.362062 4224 ts_tablet_manager.cc:1367] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:33.362951 4224 raft_consensus.cc:348] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:33.362931 4247 tablet_bootstrap.cc:654] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.363196 4224 raft_consensus.cc:374] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.363242 4224 raft_consensus.cc:729] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.363405 4224 consensus_queue.cc:260] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:33.364527 4247 tablet_bootstrap.cc:492] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:33.364611 4247 ts_tablet_manager.cc:1367] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:33.364828 4241 tablet_bootstrap.cc:492] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:33.364905 4224 ts_tablet_manager.cc:1398] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:33.364933 4241 ts_tablet_manager.cc:1367] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 14:00:33.365084 4224 tablet_bootstrap.cc:492] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:33.365398 4241 raft_consensus.cc:348] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:33.365520 4241 raft_consensus.cc:374] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.365453 4247 raft_consensus.cc:348] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:33.365551 4241 raft_consensus.cc:729] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.365561 4247 raft_consensus.cc:374] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.365594 4247 raft_consensus.cc:729] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.365669 4241 consensus_queue.cc:260] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:33.365753 4247 consensus_queue.cc:260] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:33.366048 4247 ts_tablet_manager.cc:1398] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:33.366272 4224 tablet_bootstrap.cc:654] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.367278 4224 tablet_bootstrap.cc:492] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:33.367357 4224 ts_tablet_manager.cc:1367] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:33.367516 4241 ts_tablet_manager.cc:1398] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:33.368250 4239 tablet_bootstrap.cc:492] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:33.368333 4239 ts_tablet_manager.cc:1367] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 14:00:33.368793 4239 raft_consensus.cc:348] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:33.368896 4239 raft_consensus.cc:374] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.368927 4239 raft_consensus.cc:729] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.369052 4239 consensus_queue.cc:260] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:33.369328 4239 ts_tablet_manager.cc:1398] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:33.369484 4239 tablet_bootstrap.cc:492] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:33.370265 4239 tablet_bootstrap.cc:654] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.367780 4224 raft_consensus.cc:348] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:33.370824 4224 raft_consensus.cc:374] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.370873 4224 raft_consensus.cc:729] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.371013 4224 consensus_queue.cc:260] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:33.371317 4224 ts_tablet_manager.cc:1398] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:33.372043 4239 tablet_bootstrap.cc:492] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:33.372118 4239 ts_tablet_manager.cc:1367] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:33.372467 4239 raft_consensus.cc:348] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:33.372602 4239 raft_consensus.cc:374] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.372635 4239 raft_consensus.cc:729] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.372732 4239 consensus_queue.cc:260] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:33.373152 4239 ts_tablet_manager.cc:1398] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:33.373315 4239 tablet_bootstrap.cc:492] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:33.374042 4239 tablet_bootstrap.cc:654] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.375239 4239 tablet_bootstrap.cc:492] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:33.375376 4239 ts_tablet_manager.cc:1367] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:33.375746 4239 raft_consensus.cc:348] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:33.375869 4239 raft_consensus.cc:374] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.375913 4239 raft_consensus.cc:729] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.376051 4239 consensus_queue.cc:260] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:33.376287 4239 ts_tablet_manager.cc:1398] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:33.412766 4194 raft_consensus.cc:482] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:33.412879 4194 raft_consensus.cc:504] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:33.413293 4194 leader_election.cc:288] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:33.413604 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db51059f37184061914c56e322535314" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:33.413678 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db51059f37184061914c56e322535314" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:33.413746 2886 raft_consensus.cc:2454] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:33.413808 540 raft_consensus.cc:2454] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:33.414062 3087 leader_election.cc:302] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:33.414353 4194 raft_consensus.cc:2781] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:33.414422 4194 raft_consensus.cc:482] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:33.414475 4194 raft_consensus.cc:3037] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.415309 4194 raft_consensus.cc:504] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:33.415673 4194 leader_election.cc:288] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:33.415982 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db51059f37184061914c56e322535314" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:33.416096 2886 raft_consensus.cc:3037] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.416095 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db51059f37184061914c56e322535314" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:33.416208 540 raft_consensus.cc:3037] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.416908 2886 raft_consensus.cc:2454] T db51059f37184061914c56e322535314 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:33.417075 540 raft_consensus.cc:2454] T db51059f37184061914c56e322535314 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:33.417196 3087 leader_election.cc:302] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:33.417371 4194 raft_consensus.cc:2781] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:33.417447 4194 raft_consensus.cc:686] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:33.417598 4194 consensus_queue.cc:237] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:33.419698 32731 catalog_manager.cc:5484] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:33.427990 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:33.431169 4194 raft_consensus.cc:482] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:33.431282 4194 raft_consensus.cc:504] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:33.432009 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcaf05a01194e919efca9cc08d61c8f" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:33.432155 3019 raft_consensus.cc:2454] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:33.432461 3084 leader_election.cc:302] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:33.432689 3710 raft_consensus.cc:2781] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:33.432746 3710 raft_consensus.cc:482] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:33.432716 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcaf05a01194e919efca9cc08d61c8f" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:33.432790 3710 raft_consensus.cc:3037] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.432843 2886 raft_consensus.cc:2454] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:33.433526 4194 leader_election.cc:288] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:33.433727 3284 raft_consensus.cc:1261] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.433791 405 raft_consensus.cc:1261] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.433851 3710 raft_consensus.cc:504] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:33.434178 4194 consensus_queue.cc:1035] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.434232 3710 leader_election.cc:288] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:33.434478 4254 consensus_queue.cc:1035] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.434525 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcaf05a01194e919efca9cc08d61c8f" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:33.434628 2886 raft_consensus.cc:3037] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.434896 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcaf05a01194e919efca9cc08d61c8f" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:33.435003 3019 raft_consensus.cc:3037] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.435686 2886 raft_consensus.cc:2454] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:33.435997 3019 raft_consensus.cc:2454] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:33.436491 3084 leader_election.cc:302] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:33.436638 4194 raft_consensus.cc:2781] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:33.436802 4194 raft_consensus.cc:686] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:33.436929 4194 consensus_queue.cc:237] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:33.437355 32731 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet b89e76ea92f847ca89e9fd2470455832 (table rebalancer_dynamic_table [id=b3c7d50540294e79a621956174314d15]). Expected version 1 got 0
I20240502 14:00:33.438982 32726 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet b89e76ea92f847ca89e9fd2470455832 (table rebalancer_dynamic_table [id=b3c7d50540294e79a621956174314d15]). Expected version 1 got 0
I20240502 14:00:33.439182 32732 catalog_manager.cc:5484] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:33.439779 4233 mvcc.cc:205] Tried to move back new op lower bound from 7023240943337488384 to 7023240942748217344. Current Snapshot: MvccSnapshot[applied={T|T < 7023240943337488384}]
I20240502 14:00:33.442113 4261 tablet.cc:1722] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:33.442607 4261 tablet.cc:1620] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:33.442741 32732 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet b89e76ea92f847ca89e9fd2470455832 (table rebalancer_dynamic_table [id=b3c7d50540294e79a621956174314d15]). Expected version 1 got 0
I20240502 14:00:33.443063 32726 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet b89e76ea92f847ca89e9fd2470455832 (table rebalancer_dynamic_table [id=b3c7d50540294e79a621956174314d15]). Expected version 1 got 0
I20240502 14:00:33.443065 4260 tablet.cc:1722] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:33.443351 4260 tablet.cc:1620] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:33.443801 32732 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet b89e76ea92f847ca89e9fd2470455832 (table rebalancer_dynamic_table [id=b3c7d50540294e79a621956174314d15]). Expected version 1 got 0
I20240502 14:00:33.446974 4262 tablet.cc:1722] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:33.447276 4262 tablet.cc:1620] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:33.448443 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=b3c7d50540294e79a621956174314d15] alter complete (version 1)
I20240502 14:00:33.550611 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:33.556447 4260 tablet.cc:1722] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:33.557011 4261 tablet.cc:1722] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:33.557377 4260 tablet.cc:1620] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:33.557492 4261 tablet.cc:1620] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:33.557641 4262 tablet.cc:1722] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:33.557960 4262 tablet.cc:1620] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:33.563261 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=b3c7d50540294e79a621956174314d15] alter complete (version 2)
I20240502 14:00:33.658825 3503 raft_consensus.cc:482] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:33.658948 3503 raft_consensus.cc:504] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:33.659456 3503 leader_election.cc:288] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:33.660122 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf2d71983204c3c96d112cab1004478" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:33.660274 3152 raft_consensus.cc:2454] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:33.660580 2822 leader_election.cc:302] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:33.660745 3503 raft_consensus.cc:2781] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:33.660800 3503 raft_consensus.cc:482] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:33.660861 3503 raft_consensus.cc:3037] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.662115 3503 raft_consensus.cc:504] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:33.662848 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf2d71983204c3c96d112cab1004478" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:33.662971 3152 raft_consensus.cc:3037] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.664048 3152 raft_consensus.cc:2454] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:33.664321 2822 leader_election.cc:302] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:33.664496 3956 raft_consensus.cc:2781] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:33.664705 3956 raft_consensus.cc:686] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:33.664805 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf2d71983204c3c96d112cab1004478" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:33.664876 3956 consensus_queue.cc:237] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:33.664918 672 raft_consensus.cc:2454] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:33.665453 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cf2d71983204c3c96d112cab1004478" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:33.665547 672 raft_consensus.cc:3037] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.666399 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:33.666456 672 raft_consensus.cc:2454] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:33.668174 32731 catalog_manager.cc:5484] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:33.672484 3503 leader_election.cc:288] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:33.672837 4260 tablet.cc:1722] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:33.672974 4262 tablet.cc:1722] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:33.676338 4261 tablet.cc:1722] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:33.678228 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=b3c7d50540294e79a621956174314d15] alter complete (version 3)
I20240502 14:00:33.780958 32732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:33.781116 32732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:33.783321 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet b89e76ea92f847ca89e9fd2470455832
I20240502 14:00:33.783915 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet b89e76ea92f847ca89e9fd2470455832 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:33 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:33.784056 386 tablet_service.cc:1514] Processing DeleteTablet for tablet b89e76ea92f847ca89e9fd2470455832 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:33 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:33.784291 4267 tablet_replica.cc:330] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:33.784447 4267 raft_consensus.cc:2229] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:33.784574 4267 raft_consensus.cc:2258] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:33.784667 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet b89e76ea92f847ca89e9fd2470455832 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:33 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:33.784906 4268 tablet_replica.cc:330] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:33.785074 4269 tablet_replica.cc:330] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:33.785097 4268 raft_consensus.cc:2229] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:33.785181 4267 ts_tablet_manager.cc:1875] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:33.785265 4268 raft_consensus.cc:2258] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:33.785287 4269 raft_consensus.cc:2229] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:33.785728 4269 raft_consensus.cc:2258] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:33.785797 4268 ts_tablet_manager.cc:1875] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:33.787031 4269 ts_tablet_manager.cc:1875] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:33.788237 4268 ts_tablet_manager.cc:1888] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:33.788314 4268 log.cc:1196] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/b89e76ea92f847ca89e9fd2470455832
I20240502 14:00:33.788344 4267 ts_tablet_manager.cc:1888] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:33.788447 4267 log.cc:1196] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/b89e76ea92f847ca89e9fd2470455832
I20240502 14:00:33.788669 4268 ts_tablet_manager.cc:1909] T b89e76ea92f847ca89e9fd2470455832 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:33.788733 4267 ts_tablet_manager.cc:1909] T b89e76ea92f847ca89e9fd2470455832 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:33.789357 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet b89e76ea92f847ca89e9fd2470455832 (table rebalancer_dynamic_table_new_name [id=b3c7d50540294e79a621956174314d15]) successfully deleted
I20240502 14:00:33.789466 4269 ts_tablet_manager.cc:1888] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:33.789479 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet b89e76ea92f847ca89e9fd2470455832 (table rebalancer_dynamic_table_new_name [id=b3c7d50540294e79a621956174314d15]) successfully deleted
I20240502 14:00:33.789556 4269 log.cc:1196] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/b89e76ea92f847ca89e9fd2470455832
I20240502 14:00:33.789829 4269 ts_tablet_manager.cc:1909] T b89e76ea92f847ca89e9fd2470455832 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:33.790343 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet b89e76ea92f847ca89e9fd2470455832 (table rebalancer_dynamic_table_new_name [id=b3c7d50540294e79a621956174314d15]) successfully deleted
I20240502 14:00:33.804811 4258 consensus_queue.cc:1035] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.808733 4258 consensus_queue.cc:1035] T db51059f37184061914c56e322535314 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.872584 4258 consensus_queue.cc:1035] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.876839 4254 consensus_queue.cc:1035] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.885367 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:33.890689 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 2a6d45fa60a641edabdc41ef82b0a827 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=daf78b82b0c5469cb13d189df8bee8d8]), partition=
I20240502 14:00:33.890789 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 2a6d45fa60a641edabdc41ef82b0a827 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=daf78b82b0c5469cb13d189df8bee8d8]), partition=
I20240502 14:00:33.890830 653 tablet_service.cc:1467] Processing CreateTablet for tablet 2a6d45fa60a641edabdc41ef82b0a827 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=daf78b82b0c5469cb13d189df8bee8d8]), partition=
I20240502 14:00:33.891018 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a6d45fa60a641edabdc41ef82b0a827. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.891065 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a6d45fa60a641edabdc41ef82b0a827. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.891089 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a6d45fa60a641edabdc41ef82b0a827. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.893201 4278 tablet_bootstrap.cc:492] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:33.893510 4280 tablet_bootstrap.cc:492] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:33.894436 4280 tablet_bootstrap.cc:654] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.894450 4278 tablet_bootstrap.cc:654] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.895043 4279 tablet_bootstrap.cc:492] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:33.895480 4278 tablet_bootstrap.cc:492] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:33.895568 4278 ts_tablet_manager.cc:1367] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:33.895848 4279 tablet_bootstrap.cc:654] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.896014 4278 raft_consensus.cc:348] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:33.896142 4278 raft_consensus.cc:374] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.896176 4278 raft_consensus.cc:729] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.896340 4278 consensus_queue.cc:260] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:33.896723 4278 ts_tablet_manager.cc:1398] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:33.897039 4280 tablet_bootstrap.cc:492] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:33.897114 4280 ts_tablet_manager.cc:1367] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:33.897547 4280 raft_consensus.cc:348] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:33.897662 4280 raft_consensus.cc:374] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.897729 4280 raft_consensus.cc:729] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.897825 4279 tablet_bootstrap.cc:492] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:33.897931 4279 ts_tablet_manager.cc:1367] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:33.897951 4280 consensus_queue.cc:260] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:33.898341 4280 ts_tablet_manager.cc:1398] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:33.898418 4279 raft_consensus.cc:348] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:33.898555 4279 raft_consensus.cc:374] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.898602 4279 raft_consensus.cc:729] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.898720 4279 consensus_queue.cc:260] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:33.899029 4279 ts_tablet_manager.cc:1398] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:34.015990 4139 raft_consensus.cc:482] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:34.016113 4139 raft_consensus.cc:504] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:34.016633 4139 leader_election.cc:288] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:34.017324 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6d45fa60a641edabdc41ef82b0a827" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:34.017469 3283 raft_consensus.cc:2454] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:34.018007 2952 leader_election.cc:302] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:34.018630 4139 raft_consensus.cc:2781] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:34.018689 4139 raft_consensus.cc:482] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:34.018723 4139 raft_consensus.cc:3037] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.020352 4139 raft_consensus.cc:504] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:34.020819 4139 leader_election.cc:288] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:34.020869 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6d45fa60a641edabdc41ef82b0a827" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:34.020980 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6d45fa60a641edabdc41ef82b0a827" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:34.021021 672 raft_consensus.cc:2454] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:34.021085 3283 raft_consensus.cc:3037] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.021445 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6d45fa60a641edabdc41ef82b0a827" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:34.021551 672 raft_consensus.cc:3037] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.022218 3283 raft_consensus.cc:2454] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:34.022563 672 raft_consensus.cc:2454] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:34.027935 2952 leader_election.cc:302] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:34.028131 4139 raft_consensus.cc:2781] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:34.028573 4139 raft_consensus.cc:686] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:34.029052 4139 consensus_queue.cc:237] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:34.031307 32726 catalog_manager.cc:5484] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:34.044063 3503 consensus_queue.cc:1035] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:34.047991 4265 consensus_queue.cc:1035] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:34.145833 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:34.149919 3283 raft_consensus.cc:1261] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:34.150022 672 raft_consensus.cc:1261] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:34.150372 4139 consensus_queue.cc:1035] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:34.150559 4286 consensus_queue.cc:1035] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:34.153112 32731 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 2a6d45fa60a641edabdc41ef82b0a827 (table rebalancer_dynamic_table [id=daf78b82b0c5469cb13d189df8bee8d8]). Expected version 1 got 0
I20240502 14:00:34.153573 4293 tablet.cc:1722] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:34.153934 4293 tablet.cc:1620] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:34.154937 32731 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet 2a6d45fa60a641edabdc41ef82b0a827 (table rebalancer_dynamic_table [id=daf78b82b0c5469cb13d189df8bee8d8]). Expected version 1 got 0
I20240502 14:00:34.155346 4261 tablet.cc:1722] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:34.155584 4261 tablet.cc:1620] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:34.156628 32732 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 2a6d45fa60a641edabdc41ef82b0a827 (table rebalancer_dynamic_table [id=daf78b82b0c5469cb13d189df8bee8d8]). Expected version 1 got 0
I20240502 14:00:34.157882 4294 tablet.cc:1722] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:34.158470 4294 tablet.cc:1620] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:34.159734 32731 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 2a6d45fa60a641edabdc41ef82b0a827 (table rebalancer_dynamic_table [id=daf78b82b0c5469cb13d189df8bee8d8]). Expected version 1 got 0
I20240502 14:00:34.160388 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=daf78b82b0c5469cb13d189df8bee8d8] alter complete (version 1)
I20240502 14:00:34.224391 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:34.231173 386 tablet_service.cc:1467] Processing CreateTablet for tablet be10cc2fc95c475f8bb0868229f748f4 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=051523fc8cdc4372b1911d17412f7678]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:34.231536 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be10cc2fc95c475f8bb0868229f748f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.231526 786 tablet_service.cc:1467] Processing CreateTablet for tablet 6787b4081a7847b8b0701b05936d441c (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=051523fc8cdc4372b1911d17412f7678]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:34.231853 786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6787b4081a7847b8b0701b05936d441c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.232357 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 79db2b3bbcb041f6be0d86b150a3d324 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=051523fc8cdc4372b1911d17412f7678]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:34.232627 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79db2b3bbcb041f6be0d86b150a3d324. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.233845 4295 tablet_bootstrap.cc:492] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:34.234012 2864 tablet_service.cc:1467] Processing CreateTablet for tablet 6787b4081a7847b8b0701b05936d441c (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=051523fc8cdc4372b1911d17412f7678]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:34.234181 784 tablet_service.cc:1467] Processing CreateTablet for tablet be10cc2fc95c475f8bb0868229f748f4 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=051523fc8cdc4372b1911d17412f7678]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:34.234261 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6787b4081a7847b8b0701b05936d441c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.234441 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be10cc2fc95c475f8bb0868229f748f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.234820 4295 tablet_bootstrap.cc:654] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.236086 4296 tablet_bootstrap.cc:492] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:34.236483 653 tablet_service.cc:1467] Processing CreateTablet for tablet 79db2b3bbcb041f6be0d86b150a3d324 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=051523fc8cdc4372b1911d17412f7678]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:34.236755 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79db2b3bbcb041f6be0d86b150a3d324. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.236734 3264 tablet_service.cc:1467] Processing CreateTablet for tablet be10cc2fc95c475f8bb0868229f748f4 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=051523fc8cdc4372b1911d17412f7678]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:34.237001 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be10cc2fc95c475f8bb0868229f748f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.237540 4296 tablet_bootstrap.cc:654] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.238741 4279 tablet_bootstrap.cc:492] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:34.239159 4278 tablet_bootstrap.cc:492] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:34.239733 4279 tablet_bootstrap.cc:654] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.240166 4278 tablet_bootstrap.cc:654] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.240749 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 6787b4081a7847b8b0701b05936d441c (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=051523fc8cdc4372b1911d17412f7678]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:34.241065 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6787b4081a7847b8b0701b05936d441c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.241103 4295 tablet_bootstrap.cc:492] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:34.241228 4295 ts_tablet_manager.cc:1367] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:34.241336 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 79db2b3bbcb041f6be0d86b150a3d324 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=051523fc8cdc4372b1911d17412f7678]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:34.241573 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79db2b3bbcb041f6be0d86b150a3d324. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.241771 4279 tablet_bootstrap.cc:492] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:34.241734 4295 raft_consensus.cc:348] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.241852 4279 ts_tablet_manager.cc:1367] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:34.241851 4295 raft_consensus.cc:374] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.241884 4295 raft_consensus.cc:729] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.242033 4295 consensus_queue.cc:260] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.242337 4295 ts_tablet_manager.cc:1398] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:34.242300 4279 raft_consensus.cc:348] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:34.242399 4279 raft_consensus.cc:374] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.242431 4279 raft_consensus.cc:729] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.242507 4295 tablet_bootstrap.cc:492] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:34.242575 4279 consensus_queue.cc:260] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:34.242906 4279 ts_tablet_manager.cc:1398] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:34.243593 4295 tablet_bootstrap.cc:654] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.244238 4297 tablet_bootstrap.cc:492] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:34.245348 4297 tablet_bootstrap.cc:654] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.246933 4297 tablet_bootstrap.cc:492] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:34.247053 4297 ts_tablet_manager.cc:1367] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:34.248136 4297 raft_consensus.cc:348] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:34.248292 4297 raft_consensus.cc:374] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.248353 4297 raft_consensus.cc:729] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.248544 4297 consensus_queue.cc:260] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:34.248982 4297 ts_tablet_manager.cc:1398] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:34.249300 4297 tablet_bootstrap.cc:492] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:34.250845 4278 tablet_bootstrap.cc:492] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:34.250946 4278 ts_tablet_manager.cc:1367] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 14:00:34.250986 4295 tablet_bootstrap.cc:492] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:34.251070 4295 ts_tablet_manager.cc:1367] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:34.251432 4278 raft_consensus.cc:348] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:34.251464 4295 raft_consensus.cc:348] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:34.251554 4278 raft_consensus.cc:374] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.251554 4295 raft_consensus.cc:374] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.251585 4278 raft_consensus.cc:729] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.251585 4295 raft_consensus.cc:729] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.251705 4295 consensus_queue.cc:260] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:34.251705 4278 consensus_queue.cc:260] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:34.251989 4278 ts_tablet_manager.cc:1398] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:34.252094 4295 ts_tablet_manager.cc:1398] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:34.252269 4297 tablet_bootstrap.cc:654] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.256395 4297 tablet_bootstrap.cc:492] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:34.256520 4297 ts_tablet_manager.cc:1367] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:34.257098 4297 raft_consensus.cc:348] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.257253 4297 raft_consensus.cc:374] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.257309 4297 raft_consensus.cc:729] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.257445 4297 consensus_queue.cc:260] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.257782 4297 ts_tablet_manager.cc:1398] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:34.261698 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:34.267606 4293 tablet.cc:1722] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:34.267949 4293 tablet.cc:1620] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:34.267987 4261 tablet.cc:1722] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:34.268260 4261 tablet.cc:1620] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:34.270560 4305 tablet_bootstrap.cc:492] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:34.271620 4305 tablet_bootstrap.cc:654] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.272742 4294 tablet.cc:1722] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:34.273015 4294 tablet.cc:1620] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:34.273631 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=daf78b82b0c5469cb13d189df8bee8d8] alter complete (version 2)
I20240502 14:00:34.279165 4305 tablet_bootstrap.cc:492] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:34.279271 4305 ts_tablet_manager.cc:1367] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:34.279763 4305 raft_consensus.cc:348] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:34.279911 4305 raft_consensus.cc:374] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.279958 4305 raft_consensus.cc:729] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.280100 4305 consensus_queue.cc:260] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:34.280423 4305 ts_tablet_manager.cc:1398] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:34.280596 4305 tablet_bootstrap.cc:492] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:34.281502 4305 tablet_bootstrap.cc:654] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.282855 4296 tablet_bootstrap.cc:492] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:34.282869 4305 tablet_bootstrap.cc:492] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:34.282945 4305 ts_tablet_manager.cc:1367] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:34.282980 4296 ts_tablet_manager.cc:1367] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20240502 14:00:34.283391 4305 raft_consensus.cc:348] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.283502 4305 raft_consensus.cc:374] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.283457 4296 raft_consensus.cc:348] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:34.283535 4305 raft_consensus.cc:729] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.283567 4296 raft_consensus.cc:374] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.283608 4296 raft_consensus.cc:729] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.283689 4305 consensus_queue.cc:260] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.283740 4296 consensus_queue.cc:260] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:34.284009 4305 ts_tablet_manager.cc:1398] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:34.284046 4296 ts_tablet_manager.cc:1398] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:34.377291 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:34.382541 4293 tablet.cc:1722] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:34.383124 4261 tablet.cc:1722] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:34.387434 4265 raft_consensus.cc:482] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:34.387529 4265 raft_consensus.cc:504] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:34.387905 4265 leader_election.cc:288] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:34.388334 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79db2b3bbcb041f6be0d86b150a3d324" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:34.388499 3152 raft_consensus.cc:2454] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:34.388517 4294 tablet.cc:1722] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:34.388823 2822 leader_election.cc:302] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:34.389235 4265 raft_consensus.cc:2781] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:34.389287 4265 raft_consensus.cc:482] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:34.389320 4265 raft_consensus.cc:3037] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.390316 4265 raft_consensus.cc:504] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:34.390637 4265 leader_election.cc:288] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:34.389183 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=daf78b82b0c5469cb13d189df8bee8d8] alter complete (version 3)
I20240502 14:00:34.391260 3152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79db2b3bbcb041f6be0d86b150a3d324" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:34.391387 3152 raft_consensus.cc:3037] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.392383 3152 raft_consensus.cc:2454] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:34.392796 2822 leader_election.cc:302] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:34.393173 4265 raft_consensus.cc:2781] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:34.393256 4265 raft_consensus.cc:686] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:34.393275 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79db2b3bbcb041f6be0d86b150a3d324" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:34.393385 674 raft_consensus.cc:3037] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.393375 4265 consensus_queue.cc:237] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:34.394284 674 raft_consensus.cc:2454] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:34.394546 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79db2b3bbcb041f6be0d86b150a3d324" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:34.394654 672 raft_consensus.cc:2362] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1. Re-sending same reply.
I20240502 14:00:34.395700 32732 catalog_manager.cc:5484] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:34.496731 32730 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:34.496872 32730 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:34.498953 32730 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 2a6d45fa60a641edabdc41ef82b0a827
I20240502 14:00:34.499523 2998 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a6d45fa60a641edabdc41ef82b0a827 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:34 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:34.499557 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a6d45fa60a641edabdc41ef82b0a827 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:34 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:34.499698 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a6d45fa60a641edabdc41ef82b0a827 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:34 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:34.499943 4311 tablet_replica.cc:330] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:34.499951 4312 tablet_replica.cc:330] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:34.499980 4313 tablet_replica.cc:330] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:34.500088 4313 raft_consensus.cc:2229] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:34.500082 4312 raft_consensus.cc:2229] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:34.500090 4311 raft_consensus.cc:2229] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:34.500202 4313 raft_consensus.cc:2258] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:34.500202 4312 raft_consensus.cc:2258] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:34.500295 4311 raft_consensus.cc:2258] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:34.500793 4311 ts_tablet_manager.cc:1875] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:34.500854 4312 ts_tablet_manager.cc:1875] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:34.500854 4313 ts_tablet_manager.cc:1875] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:34.502733 4311 ts_tablet_manager.cc:1888] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:34.502838 4311 log.cc:1196] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/2a6d45fa60a641edabdc41ef82b0a827
I20240502 14:00:34.503010 4312 ts_tablet_manager.cc:1888] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:34.503013 4313 ts_tablet_manager.cc:1888] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:34.503085 4313 log.cc:1196] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/2a6d45fa60a641edabdc41ef82b0a827
I20240502 14:00:34.503110 4312 log.cc:1196] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/2a6d45fa60a641edabdc41ef82b0a827
I20240502 14:00:34.503152 4311 ts_tablet_manager.cc:1909] T 2a6d45fa60a641edabdc41ef82b0a827 P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:34.503401 4312 ts_tablet_manager.cc:1909] T 2a6d45fa60a641edabdc41ef82b0a827 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:34.503445 4313 ts_tablet_manager.cc:1909] T 2a6d45fa60a641edabdc41ef82b0a827 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:34.503885 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 2a6d45fa60a641edabdc41ef82b0a827 (table rebalancer_dynamic_table_new_name [id=daf78b82b0c5469cb13d189df8bee8d8]) successfully deleted
I20240502 14:00:34.504065 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 2a6d45fa60a641edabdc41ef82b0a827 (table rebalancer_dynamic_table_new_name [id=daf78b82b0c5469cb13d189df8bee8d8]) successfully deleted
I20240502 14:00:34.504266 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 2a6d45fa60a641edabdc41ef82b0a827 (table rebalancer_dynamic_table_new_name [id=daf78b82b0c5469cb13d189df8bee8d8]) successfully deleted
I20240502 14:00:34.527029 4231 raft_consensus.cc:482] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:34.527158 4231 raft_consensus.cc:504] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.527540 4231 leader_election.cc:288] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:34.528009 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6787b4081a7847b8b0701b05936d441c" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:34.528159 2886 raft_consensus.cc:2454] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:34.528155 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6787b4081a7847b8b0701b05936d441c" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:34.528319 3151 raft_consensus.cc:2454] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:34.528460 741 leader_election.cc:302] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:34.528630 4231 raft_consensus.cc:2781] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:34.528689 4231 raft_consensus.cc:482] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:34.528743 4231 raft_consensus.cc:3037] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.529644 4231 raft_consensus.cc:504] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.529963 4231 leader_election.cc:288] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:34.530297 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6787b4081a7847b8b0701b05936d441c" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:34.530372 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6787b4081a7847b8b0701b05936d441c" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:34.530416 3151 raft_consensus.cc:3037] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.530488 2886 raft_consensus.cc:3037] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.531226 3151 raft_consensus.cc:2454] T 6787b4081a7847b8b0701b05936d441c P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:34.531423 2886 raft_consensus.cc:2454] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:34.531728 741 leader_election.cc:302] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:34.531942 4231 raft_consensus.cc:2781] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:34.532032 4231 raft_consensus.cc:686] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:34.532169 4231 consensus_queue.cc:237] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.534096 32730 catalog_manager.cc:5484] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:34.558393 4274 raft_consensus.cc:482] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:34.558529 4274 raft_consensus.cc:504] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:34.558979 4274 leader_election.cc:288] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:34.559276 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be10cc2fc95c475f8bb0868229f748f4" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:34.559428 409 raft_consensus.cc:2454] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:34.559867 3218 leader_election.cc:302] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:34.560052 4274 raft_consensus.cc:2781] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:34.560120 4274 raft_consensus.cc:482] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:34.560148 4274 raft_consensus.cc:3037] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.560992 4274 raft_consensus.cc:504] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:34.561357 4274 leader_election.cc:288] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:34.561686 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be10cc2fc95c475f8bb0868229f748f4" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:34.561812 409 raft_consensus.cc:3037] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.562754 409 raft_consensus.cc:2454] T be10cc2fc95c475f8bb0868229f748f4 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:34.563022 3218 leader_election.cc:302] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:34.563037 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be10cc2fc95c475f8bb0868229f748f4" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:34.563129 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be10cc2fc95c475f8bb0868229f748f4" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:34.563161 804 raft_consensus.cc:2454] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:34.563205 4274 raft_consensus.cc:2781] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:34.563261 805 raft_consensus.cc:3037] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.563371 4274 raft_consensus.cc:686] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:34.563519 4274 consensus_queue.cc:237] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:34.564244 805 raft_consensus.cc:2454] T be10cc2fc95c475f8bb0868229f748f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:34.565753 32730 catalog_manager.cc:5484] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:34.600840 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:34.606678 3264 tablet_service.cc:1467] Processing CreateTablet for tablet ad30ba64e3fb49d6aab470aebca25b77 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1e2e9e5f4bfb4e368f25d5ea59208f96]), partition=
I20240502 14:00:34.606690 2998 tablet_service.cc:1467] Processing CreateTablet for tablet ad30ba64e3fb49d6aab470aebca25b77 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1e2e9e5f4bfb4e368f25d5ea59208f96]), partition=
I20240502 14:00:34.606987 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad30ba64e3fb49d6aab470aebca25b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.606986 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad30ba64e3fb49d6aab470aebca25b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.608702 4278 tablet_bootstrap.cc:492] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:34.608965 4319 tablet_bootstrap.cc:492] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:34.609638 2864 tablet_service.cc:1467] Processing CreateTablet for tablet ad30ba64e3fb49d6aab470aebca25b77 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1e2e9e5f4bfb4e368f25d5ea59208f96]), partition=
I20240502 14:00:34.609714 4278 tablet_bootstrap.cc:654] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.609783 4319 tablet_bootstrap.cc:654] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.609876 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad30ba64e3fb49d6aab470aebca25b77. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:34.611029 4319 tablet_bootstrap.cc:492] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:34.611117 4319 ts_tablet_manager.cc:1367] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:34.611168 4278 tablet_bootstrap.cc:492] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:34.611249 4278 ts_tablet_manager.cc:1367] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:34.611713 4278 raft_consensus.cc:348] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.611862 4278 raft_consensus.cc:374] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.611898 4278 raft_consensus.cc:729] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.612027 4278 consensus_queue.cc:260] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.612125 4297 tablet_bootstrap.cc:492] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:34.612299 4278 ts_tablet_manager.cc:1398] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:34.612916 4297 tablet_bootstrap.cc:654] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:34.613960 4297 tablet_bootstrap.cc:492] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:34.614039 4297 ts_tablet_manager.cc:1367] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:34.614392 4297 raft_consensus.cc:348] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.614488 4297 raft_consensus.cc:374] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.614511 4297 raft_consensus.cc:729] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.614614 4297 consensus_queue.cc:260] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.614897 4297 ts_tablet_manager.cc:1398] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:34.614840 4319 raft_consensus.cc:348] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.614949 4319 raft_consensus.cc:374] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:34.614982 4319 raft_consensus.cc:729] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:34.615092 4319 consensus_queue.cc:260] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.615950 4319 ts_tablet_manager.cc:1398] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:34.722450 3503 raft_consensus.cc:482] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:34.722574 3503 raft_consensus.cc:504] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.722944 3503 leader_election.cc:288] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:34.723352 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad30ba64e3fb49d6aab470aebca25b77" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:34.723372 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad30ba64e3fb49d6aab470aebca25b77" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:34.723495 3019 raft_consensus.cc:2454] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:34.723506 3283 raft_consensus.cc:2454] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:34.723778 2820 leader_election.cc:302] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:34.723951 3503 raft_consensus.cc:2781] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:34.724016 3503 raft_consensus.cc:482] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:34.724059 3503 raft_consensus.cc:3037] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.724862 3503 raft_consensus.cc:504] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.725179 3503 leader_election.cc:288] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:34.725528 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad30ba64e3fb49d6aab470aebca25b77" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:34.725536 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad30ba64e3fb49d6aab470aebca25b77" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:34.725636 3019 raft_consensus.cc:3037] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.725648 3283 raft_consensus.cc:3037] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:34.726356 3019 raft_consensus.cc:2454] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:34.726436 3283 raft_consensus.cc:2454] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:34.726686 2819 leader_election.cc:302] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:34.726883 3503 raft_consensus.cc:2781] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:34.726958 3503 raft_consensus.cc:686] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:34.727102 3503 consensus_queue.cc:237] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:34.729153 32732 catalog_manager.cc:5484] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:34.856014 3503 consensus_queue.cc:1035] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:34.860034 3956 consensus_queue.cc:1035] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:34.861404 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:34.865792 3283 raft_consensus.cc:1261] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:34.865913 3019 raft_consensus.cc:1261] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:34.866163 3503 consensus_queue.cc:1035] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:34.866331 3956 consensus_queue.cc:1035] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:34.868671 32732 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet ad30ba64e3fb49d6aab470aebca25b77 (table rebalancer_dynamic_table [id=1e2e9e5f4bfb4e368f25d5ea59208f96]). Expected version 1 got 0
I20240502 14:00:34.869241 4331 tablet.cc:1722] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:34.869529 4331 tablet.cc:1620] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:34.870157 4261 tablet.cc:1722] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:34.870425 4261 tablet.cc:1620] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:34.870842 32726 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet ad30ba64e3fb49d6aab470aebca25b77 (table rebalancer_dynamic_table [id=1e2e9e5f4bfb4e368f25d5ea59208f96]). Expected version 1 got 0
I20240502 14:00:34.871914 4293 tablet.cc:1722] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:34.872225 4293 tablet.cc:1620] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:34.872679 32731 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet ad30ba64e3fb49d6aab470aebca25b77 (table rebalancer_dynamic_table [id=1e2e9e5f4bfb4e368f25d5ea59208f96]). Expected version 1 got 0
I20240502 14:00:34.875399 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=1e2e9e5f4bfb4e368f25d5ea59208f96] alter complete (version 1)
I20240502 14:00:34.950098 4274 consensus_queue.cc:1035] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:34.953476 4274 consensus_queue.cc:1035] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:34.977653 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:34.982821 4331 tablet.cc:1722] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:34.983179 4331 tablet.cc:1620] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:34.983213 4261 tablet.cc:1722] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:34.983510 4261 tablet.cc:1620] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:34.985124 4293 tablet.cc:1722] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:34.985466 4293 tablet.cc:1620] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:34.986742 4231 consensus_queue.cc:1035] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:34.989778 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=1e2e9e5f4bfb4e368f25d5ea59208f96] alter complete (version 2)
I20240502 14:00:34.992556 4338 consensus_queue.cc:1035] T 6787b4081a7847b8b0701b05936d441c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:35.093504 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:35.098917 4331 tablet.cc:1722] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:35.099525 4293 tablet.cc:1722] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:35.099686 4261 tablet.cc:1722] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:35.104214 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=1e2e9e5f4bfb4e368f25d5ea59208f96] alter complete (version 3)
I20240502 14:00:35.118229 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:35.123965 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc8d5ad25334f0a8c2efd17b5779330 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=274e7365daae4e2dba024f22f983d794]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.124001 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc8d5ad25334f0a8c2efd17b5779330 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=274e7365daae4e2dba024f22f983d794]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.124284 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc8d5ad25334f0a8c2efd17b5779330. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.124302 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc8d5ad25334f0a8c2efd17b5779330. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.124836 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 75c2963776af487b9ff7ee3004fcaa07 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=274e7365daae4e2dba024f22f983d794]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.125095 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75c2963776af487b9ff7ee3004fcaa07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.126120 4341 tablet_bootstrap.cc:492] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:35.126739 4342 tablet_bootstrap.cc:492] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:35.126770 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 75c2963776af487b9ff7ee3004fcaa07 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=274e7365daae4e2dba024f22f983d794]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.127008 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75c2963776af487b9ff7ee3004fcaa07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.127329 4341 tablet_bootstrap.cc:654] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.127748 4342 tablet_bootstrap.cc:654] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.128755 4342 tablet_bootstrap.cc:492] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:35.128840 4342 ts_tablet_manager.cc:1367] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:35.129283 4342 raft_consensus.cc:348] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:35.129395 4342 raft_consensus.cc:374] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.129429 4342 raft_consensus.cc:729] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.129542 4342 consensus_queue.cc:260] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:35.129619 2864 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc8d5ad25334f0a8c2efd17b5779330 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=274e7365daae4e2dba024f22f983d794]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.129643 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 413dd188108549d6b19b7d13bbfc3779 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=274e7365daae4e2dba024f22f983d794]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.129835 4342 ts_tablet_manager.cc:1398] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:35.129886 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc8d5ad25334f0a8c2efd17b5779330. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.130009 4342 tablet_bootstrap.cc:492] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:35.130133 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 413dd188108549d6b19b7d13bbfc3779. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.130144 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 413dd188108549d6b19b7d13bbfc3779 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=274e7365daae4e2dba024f22f983d794]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.130381 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 413dd188108549d6b19b7d13bbfc3779. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.131579 4342 tablet_bootstrap.cc:654] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.128376 4343 tablet_bootstrap.cc:492] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:35.132344 522 tablet_service.cc:1467] Processing CreateTablet for tablet 75c2963776af487b9ff7ee3004fcaa07 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=274e7365daae4e2dba024f22f983d794]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.132603 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75c2963776af487b9ff7ee3004fcaa07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.132645 4346 tablet_bootstrap.cc:492] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:35.133282 4343 tablet_bootstrap.cc:654] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.133536 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 413dd188108549d6b19b7d13bbfc3779 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=274e7365daae4e2dba024f22f983d794]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.133757 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 413dd188108549d6b19b7d13bbfc3779. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.134495 4346 tablet_bootstrap.cc:654] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.134821 4349 tablet_bootstrap.cc:492] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:35.135574 4342 tablet_bootstrap.cc:492] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:35.135634 4346 tablet_bootstrap.cc:492] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:35.135661 4342 ts_tablet_manager.cc:1367] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:35.135702 4346 ts_tablet_manager.cc:1367] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:35.136142 4342 raft_consensus.cc:348] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:35.136200 4346 raft_consensus.cc:348] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:35.136265 4342 raft_consensus.cc:374] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.136286 4346 raft_consensus.cc:374] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.136302 4342 raft_consensus.cc:729] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.136320 4346 raft_consensus.cc:729] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.136415 4343 tablet_bootstrap.cc:492] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:35.136421 4342 consensus_queue.cc:260] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:35.136423 4346 consensus_queue.cc:260] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:35.136514 4343 ts_tablet_manager.cc:1367] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:35.136713 4342 ts_tablet_manager.cc:1398] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:35.136734 4346 ts_tablet_manager.cc:1398] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:35.136875 4346 tablet_bootstrap.cc:492] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:35.136979 4343 raft_consensus.cc:348] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:35.137077 4343 raft_consensus.cc:374] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.137106 4343 raft_consensus.cc:729] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.137245 4343 consensus_queue.cc:260] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:35.137581 4343 ts_tablet_manager.cc:1398] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:35.137694 4346 tablet_bootstrap.cc:654] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.138324 4349 tablet_bootstrap.cc:654] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.139809 4349 tablet_bootstrap.cc:492] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:35.139957 4349 ts_tablet_manager.cc:1367] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:35.140386 4349 raft_consensus.cc:348] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:35.140503 4349 raft_consensus.cc:374] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.140583 4349 raft_consensus.cc:729] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.140719 4349 consensus_queue.cc:260] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:35.141134 4349 ts_tablet_manager.cc:1398] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:35.146860 4342 tablet_bootstrap.cc:492] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:35.147435 4341 tablet_bootstrap.cc:492] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:35.147502 4341 ts_tablet_manager.cc:1367] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.001s
I20240502 14:00:35.147670 4346 tablet_bootstrap.cc:492] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:35.147751 4346 ts_tablet_manager.cc:1367] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240502 14:00:35.147918 4341 raft_consensus.cc:348] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:35.148042 4341 raft_consensus.cc:374] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.148079 4341 raft_consensus.cc:729] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.148090 4342 tablet_bootstrap.cc:654] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.148203 4341 consensus_queue.cc:260] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:35.148344 4346 raft_consensus.cc:348] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:35.148473 4346 raft_consensus.cc:374] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.148506 4346 raft_consensus.cc:729] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.148502 4341 ts_tablet_manager.cc:1398] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:35.148648 4341 tablet_bootstrap.cc:492] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:35.148680 4346 consensus_queue.cc:260] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:35.148958 4346 ts_tablet_manager.cc:1398] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:35.149734 4341 tablet_bootstrap.cc:654] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.150123 4342 tablet_bootstrap.cc:492] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:35.150183 4342 ts_tablet_manager.cc:1367] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:00:35.150489 4342 raft_consensus.cc:348] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:35.150570 4342 raft_consensus.cc:374] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.150589 4342 raft_consensus.cc:729] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.150682 4342 consensus_queue.cc:260] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:35.150985 4342 ts_tablet_manager.cc:1398] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:35.151058 4341 tablet_bootstrap.cc:492] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:35.151180 4341 ts_tablet_manager.cc:1367] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:35.151687 4341 raft_consensus.cc:348] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:35.151901 4341 raft_consensus.cc:374] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.151998 4341 raft_consensus.cc:729] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.152176 4341 consensus_queue.cc:260] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:35.152532 4341 ts_tablet_manager.cc:1398] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:35.206080 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:35.206262 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:35.208464 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet ad30ba64e3fb49d6aab470aebca25b77
I20240502 14:00:35.209036 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet ad30ba64e3fb49d6aab470aebca25b77 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:35 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:35.209074 2998 tablet_service.cc:1514] Processing DeleteTablet for tablet ad30ba64e3fb49d6aab470aebca25b77 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:35 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:35.209379 4355 tablet_replica.cc:330] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:35.209539 4356 tablet_replica.cc:330] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:35.209582 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet ad30ba64e3fb49d6aab470aebca25b77 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:35 UTC) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:35.209642 4355 raft_consensus.cc:2229] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:35.209719 4356 raft_consensus.cc:2229] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:35.209779 4355 raft_consensus.cc:2258] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:35.209859 4356 raft_consensus.cc:2258] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:35.210331 4357 tablet_replica.cc:330] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:35.210517 4356 ts_tablet_manager.cc:1875] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:35.210520 4357 raft_consensus.cc:2229] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:35.210750 4357 raft_consensus.cc:2258] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:35.210757 4355 ts_tablet_manager.cc:1875] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:35.211211 4357 ts_tablet_manager.cc:1875] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:35.213364 4356 ts_tablet_manager.cc:1888] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:35.213364 4355 ts_tablet_manager.cc:1888] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:35.213390 4357 ts_tablet_manager.cc:1888] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:35.213444 4355 log.cc:1196] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/ad30ba64e3fb49d6aab470aebca25b77
I20240502 14:00:35.213492 4356 log.cc:1196] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/ad30ba64e3fb49d6aab470aebca25b77
I20240502 14:00:35.213507 4357 log.cc:1196] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/ad30ba64e3fb49d6aab470aebca25b77
I20240502 14:00:35.213815 4356 ts_tablet_manager.cc:1909] T ad30ba64e3fb49d6aab470aebca25b77 P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:35.213825 4357 ts_tablet_manager.cc:1909] T ad30ba64e3fb49d6aab470aebca25b77 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting consensus metadata
I20240502 14:00:35.214515 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet ad30ba64e3fb49d6aab470aebca25b77 (table rebalancer_dynamic_table_new_name [id=1e2e9e5f4bfb4e368f25d5ea59208f96]) successfully deleted
I20240502 14:00:35.214552 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet ad30ba64e3fb49d6aab470aebca25b77 (table rebalancer_dynamic_table_new_name [id=1e2e9e5f4bfb4e368f25d5ea59208f96]) successfully deleted
I20240502 14:00:35.214893 4355 ts_tablet_manager.cc:1909] T ad30ba64e3fb49d6aab470aebca25b77 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:35.215540 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet ad30ba64e3fb49d6aab470aebca25b77 (table rebalancer_dynamic_table_new_name [id=1e2e9e5f4bfb4e368f25d5ea59208f96]) successfully deleted
I20240502 14:00:35.285223 4139 raft_consensus.cc:482] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:35.285336 4139 raft_consensus.cc:504] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:35.285682 4139 leader_election.cc:288] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:35.286015 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75c2963776af487b9ff7ee3004fcaa07" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:35.286064 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75c2963776af487b9ff7ee3004fcaa07" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:35.286144 3283 raft_consensus.cc:2454] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:35.286233 540 raft_consensus.cc:2454] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:35.286432 2952 leader_election.cc:302] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:35.286586 4139 raft_consensus.cc:2781] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:35.286633 4139 raft_consensus.cc:482] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:35.286679 4139 raft_consensus.cc:3037] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.287602 4139 raft_consensus.cc:504] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:35.288182 4139 leader_election.cc:288] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:35.288272 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75c2963776af487b9ff7ee3004fcaa07" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:35.288343 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75c2963776af487b9ff7ee3004fcaa07" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:35.288389 540 raft_consensus.cc:3037] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.288439 3283 raft_consensus.cc:3037] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.289155 3283 raft_consensus.cc:2454] T 75c2963776af487b9ff7ee3004fcaa07 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:35.289155 540 raft_consensus.cc:2454] T 75c2963776af487b9ff7ee3004fcaa07 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:35.289428 2952 leader_election.cc:302] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:35.289490 4194 raft_consensus.cc:482] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:35.289579 4139 raft_consensus.cc:2781] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:35.289593 4194 raft_consensus.cc:504] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:35.289726 4139 raft_consensus.cc:686] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:35.289867 4139 consensus_queue.cc:237] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:35.289955 4194 leader_election.cc:288] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:35.290438 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc8d5ad25334f0a8c2efd17b5779330" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:35.290562 3019 raft_consensus.cc:2454] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:35.290819 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc8d5ad25334f0a8c2efd17b5779330" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:35.290865 3084 leader_election.cc:302] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:35.290944 2884 raft_consensus.cc:2454] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:35.291023 4194 raft_consensus.cc:2781] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:35.291064 4194 raft_consensus.cc:482] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:35.291098 4194 raft_consensus.cc:3037] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.292284 4194 raft_consensus.cc:504] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:35.292631 4194 leader_election.cc:288] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:35.292913 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc8d5ad25334f0a8c2efd17b5779330" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:35.292939 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc8d5ad25334f0a8c2efd17b5779330" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:35.292860 32731 catalog_manager.cc:5484] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.293035 3019 raft_consensus.cc:3037] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.293035 2884 raft_consensus.cc:3037] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.293896 3019 raft_consensus.cc:2454] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:35.293896 2884 raft_consensus.cc:2454] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:35.294162 3087 leader_election.cc:302] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:35.294319 4194 raft_consensus.cc:2781] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:35.294448 4194 raft_consensus.cc:686] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:35.294572 4194 consensus_queue.cc:237] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:35.296591 32731 catalog_manager.cc:5484] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.296979 4139 raft_consensus.cc:482] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:35.297072 4139 raft_consensus.cc:504] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:35.297377 4139 leader_election.cc:288] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:35.297755 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "413dd188108549d6b19b7d13bbfc3779" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:35.297780 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "413dd188108549d6b19b7d13bbfc3779" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:35.297878 2884 raft_consensus.cc:2454] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:35.297897 3151 raft_consensus.cc:2454] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:35.298173 2954 leader_election.cc:302] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:35.298331 4139 raft_consensus.cc:2781] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:35.298373 4139 raft_consensus.cc:482] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:35.298399 4139 raft_consensus.cc:3037] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.299105 4139 raft_consensus.cc:504] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:35.299374 4139 leader_election.cc:288] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:35.299681 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "413dd188108549d6b19b7d13bbfc3779" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:35.299741 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "413dd188108549d6b19b7d13bbfc3779" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:35.299813 2884 raft_consensus.cc:3037] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.299862 3151 raft_consensus.cc:3037] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.300704 2884 raft_consensus.cc:2454] T 413dd188108549d6b19b7d13bbfc3779 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:35.300726 3151 raft_consensus.cc:2454] T 413dd188108549d6b19b7d13bbfc3779 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:35.300990 2954 leader_election.cc:302] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:35.301151 4139 raft_consensus.cc:2781] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:35.301265 4139 raft_consensus.cc:686] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:35.301394 4139 consensus_queue.cc:237] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:35.303242 32731 catalog_manager.cc:5484] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.310463 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:35.315188 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7208bb09f64d4d9011775ac93d37d7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bbd44093088f4096b3f5e67abfaa184c]), partition=
I20240502 14:00:35.315249 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7208bb09f64d4d9011775ac93d37d7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bbd44093088f4096b3f5e67abfaa184c]), partition=
I20240502 14:00:35.315372 386 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7208bb09f64d4d9011775ac93d37d7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bbd44093088f4096b3f5e67abfaa184c]), partition=
I20240502 14:00:35.315464 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7208bb09f64d4d9011775ac93d37d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.315483 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7208bb09f64d4d9011775ac93d37d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.315596 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7208bb09f64d4d9011775ac93d37d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.317790 4346 tablet_bootstrap.cc:492] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:35.317939 4343 tablet_bootstrap.cc:492] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:35.318794 4346 tablet_bootstrap.cc:654] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.319047 4343 tablet_bootstrap.cc:654] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.319793 4365 tablet_bootstrap.cc:492] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:35.320035 4346 tablet_bootstrap.cc:492] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:35.320125 4346 ts_tablet_manager.cc:1367] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 14:00:35.320487 4346 raft_consensus.cc:348] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:35.320581 4346 raft_consensus.cc:374] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.320619 4346 raft_consensus.cc:729] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.320735 4346 consensus_queue.cc:260] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:35.320945 4365 tablet_bootstrap.cc:654] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.321023 4346 ts_tablet_manager.cc:1398] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:35.321702 4343 tablet_bootstrap.cc:492] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:35.321791 4343 ts_tablet_manager.cc:1367] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:35.322240 4365 tablet_bootstrap.cc:492] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:35.322369 4365 ts_tablet_manager.cc:1367] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:35.322364 4343 raft_consensus.cc:348] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:35.322492 4343 raft_consensus.cc:374] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.322535 4343 raft_consensus.cc:729] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.322664 4343 consensus_queue.cc:260] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:35.322986 4343 ts_tablet_manager.cc:1398] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:35.322798 4365 raft_consensus.cc:348] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:35.323418 4365 raft_consensus.cc:374] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.323504 4365 raft_consensus.cc:729] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.323643 4365 consensus_queue.cc:260] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:35.323959 4365 ts_tablet_manager.cc:1398] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:35.394565 4274 raft_consensus.cc:482] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:35.394681 4274 raft_consensus.cc:504] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:35.395056 4274 leader_election.cc:288] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:35.395416 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7208bb09f64d4d9011775ac93d37d7" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:35.395545 2884 raft_consensus.cc:2454] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:35.395550 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7208bb09f64d4d9011775ac93d37d7" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:35.395699 409 raft_consensus.cc:2454] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:35.395881 3219 leader_election.cc:302] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:35.396042 4274 raft_consensus.cc:2781] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:35.396113 4274 raft_consensus.cc:482] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:35.396165 4274 raft_consensus.cc:3037] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.397112 4274 raft_consensus.cc:504] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:35.397418 4274 leader_election.cc:288] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:35.397709 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7208bb09f64d4d9011775ac93d37d7" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:35.397747 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7208bb09f64d4d9011775ac93d37d7" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:35.397830 2884 raft_consensus.cc:3037] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.397840 409 raft_consensus.cc:3037] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.398767 2884 raft_consensus.cc:2454] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:35.398767 409 raft_consensus.cc:2454] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:35.399061 3218 leader_election.cc:302] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:35.399327 4274 raft_consensus.cc:2781] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:35.399397 4274 raft_consensus.cc:686] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:35.399539 4274 consensus_queue.cc:237] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:35.401453 32731 catalog_manager.cc:5484] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.516882 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:35.520615 409 raft_consensus.cc:1261] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:35.520650 2884 raft_consensus.cc:1261] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:35.521093 4370 consensus_queue.cc:1035] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:35.521485 4317 consensus_queue.cc:1035] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:35.523615 4261 tablet.cc:1722] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:35.523957 4261 tablet.cc:1620] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:35.525141 4331 tablet.cc:1722] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:35.525461 4331 tablet.cc:1620] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:35.526021 32732 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet 8d7208bb09f64d4d9011775ac93d37d7 (table rebalancer_dynamic_table [id=bbd44093088f4096b3f5e67abfaa184c]). Expected version 1 got 0
I20240502 14:00:35.527190 32731 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet 8d7208bb09f64d4d9011775ac93d37d7 (table rebalancer_dynamic_table [id=bbd44093088f4096b3f5e67abfaa184c]). Expected version 1 got 0
I20240502 14:00:35.530021 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=bbd44093088f4096b3f5e67abfaa184c] alter complete (version 1)
I20240502 14:00:35.532055 4374 tablet.cc:1722] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:35.532588 32730 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet 8d7208bb09f64d4d9011775ac93d37d7 (table rebalancer_dynamic_table [id=bbd44093088f4096b3f5e67abfaa184c]). Expected version 1 got 0
I20240502 14:00:35.532469 4374 tablet.cc:1620] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:35.631659 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:35.637177 4261 tablet.cc:1722] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:35.637547 4261 tablet.cc:1620] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:35.637487 4374 tablet.cc:1722] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:35.637784 4374 tablet.cc:1620] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:35.638170 4331 tablet.cc:1722] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:35.638497 4331 tablet.cc:1620] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:35.642894 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=bbd44093088f4096b3f5e67abfaa184c] alter complete (version 2)
I20240502 14:00:35.746107 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:35.749554 4139 consensus_queue.cc:1035] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:35.752813 4362 consensus_queue.cc:1035] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:35.753863 4360 consensus_queue.cc:1035] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:35.754374 4374 tablet.cc:1722] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:35.754501 4261 tablet.cc:1722] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:35.757830 4331 tablet.cc:1722] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:35.762135 4194 consensus_queue.cc:1035] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:35.764809 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=bbd44093088f4096b3f5e67abfaa184c] alter complete (version 3)
I20240502 14:00:35.820354 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:35.826165 3264 tablet_service.cc:1467] Processing CreateTablet for tablet c8e843cd7c984ebfb532fec67f97e249 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=2b22704f8c7e447d9b7e1edd7429f8e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.826385 2866 tablet_service.cc:1467] Processing CreateTablet for tablet c8e843cd7c984ebfb532fec67f97e249 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=2b22704f8c7e447d9b7e1edd7429f8e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.826507 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8e843cd7c984ebfb532fec67f97e249. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.826695 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8e843cd7c984ebfb532fec67f97e249. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.826874 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 0199aa852916433dafaf8a08720ca6d3 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=2b22704f8c7e447d9b7e1edd7429f8e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.827121 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0199aa852916433dafaf8a08720ca6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.828909 654 tablet_service.cc:1467] Processing CreateTablet for tablet 618c1b9af9504c03bd250d88c3a2b0ba (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=2b22704f8c7e447d9b7e1edd7429f8e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.828941 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 0199aa852916433dafaf8a08720ca6d3 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=2b22704f8c7e447d9b7e1edd7429f8e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.829187 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 618c1b9af9504c03bd250d88c3a2b0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.829216 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0199aa852916433dafaf8a08720ca6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.830767 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 618c1b9af9504c03bd250d88c3a2b0ba (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=2b22704f8c7e447d9b7e1edd7429f8e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.830866 4378 consensus_queue.cc:1035] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:35.831010 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 618c1b9af9504c03bd250d88c3a2b0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.831599 653 tablet_service.cc:1467] Processing CreateTablet for tablet c8e843cd7c984ebfb532fec67f97e249 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=2b22704f8c7e447d9b7e1edd7429f8e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.831854 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8e843cd7c984ebfb532fec67f97e249. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.831907 4385 tablet_bootstrap.cc:492] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:35.832207 4386 tablet_bootstrap.cc:492] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:35.832836 4385 tablet_bootstrap.cc:654] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.833204 4386 tablet_bootstrap.cc:654] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.833935 4385 tablet_bootstrap.cc:492] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:35.834014 4385 ts_tablet_manager.cc:1367] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:35.834333 4386 tablet_bootstrap.cc:492] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:35.834406 4386 ts_tablet_manager.cc:1367] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:35.834441 4385 raft_consensus.cc:348] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.834548 4385 raft_consensus.cc:374] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.834576 4385 raft_consensus.cc:729] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.834689 4385 consensus_queue.cc:260] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.834794 4384 tablet_bootstrap.cc:492] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:35.834894 4386 raft_consensus.cc:348] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:35.834996 4386 raft_consensus.cc:374] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.835028 4386 raft_consensus.cc:729] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.835033 4385 ts_tablet_manager.cc:1398] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:35.835186 4385 tablet_bootstrap.cc:492] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:35.835144 4386 consensus_queue.cc:260] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:35.835393 4386 ts_tablet_manager.cc:1398] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:35.835414 784 tablet_service.cc:1467] Processing CreateTablet for tablet 0199aa852916433dafaf8a08720ca6d3 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=2b22704f8c7e447d9b7e1edd7429f8e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.835685 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0199aa852916433dafaf8a08720ca6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.836539 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 618c1b9af9504c03bd250d88c3a2b0ba (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=2b22704f8c7e447d9b7e1edd7429f8e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:35.836810 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 618c1b9af9504c03bd250d88c3a2b0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.841552 4384 tablet_bootstrap.cc:654] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.842984 4360 consensus_queue.cc:1035] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:35.844733 4391 tablet_bootstrap.cc:492] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:35.845955 4391 tablet_bootstrap.cc:654] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.845624 4387 tablet_bootstrap.cc:492] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:35.848940 4385 tablet_bootstrap.cc:654] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.850098 4384 tablet_bootstrap.cc:492] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:35.850221 4384 ts_tablet_manager.cc:1367] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:00:35.850298 4387 tablet_bootstrap.cc:654] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.850746 4384 raft_consensus.cc:348] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.850868 4384 raft_consensus.cc:374] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.850914 4384 raft_consensus.cc:729] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.851052 4384 consensus_queue.cc:260] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.851332 4384 ts_tablet_manager.cc:1398] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:35.851531 4384 tablet_bootstrap.cc:492] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:35.852424 4387 tablet_bootstrap.cc:492] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:35.852494 4384 tablet_bootstrap.cc:654] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.852559 4387 ts_tablet_manager.cc:1367] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:35.853070 4387 raft_consensus.cc:348] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.853639 4385 tablet_bootstrap.cc:492] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:35.853737 4385 ts_tablet_manager.cc:1367] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 14:00:35.853844 4387 raft_consensus.cc:374] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.853921 4387 raft_consensus.cc:729] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.854089 4387 consensus_queue.cc:260] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.854251 4385 raft_consensus.cc:348] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.854377 4385 raft_consensus.cc:374] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.854411 4385 raft_consensus.cc:729] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.854528 4385 consensus_queue.cc:260] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.854820 4385 ts_tablet_manager.cc:1398] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:35.855417 4387 ts_tablet_manager.cc:1398] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:35.855590 4384 tablet_bootstrap.cc:492] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:35.855679 4384 ts_tablet_manager.cc:1367] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:35.855685 4387 tablet_bootstrap.cc:492] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:35.856155 4384 raft_consensus.cc:348] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:35.856298 4384 raft_consensus.cc:374] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.856338 4384 raft_consensus.cc:729] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.856472 4384 consensus_queue.cc:260] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:35.856753 4384 ts_tablet_manager.cc:1398] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:35.856922 4384 tablet_bootstrap.cc:492] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:35.857563 4387 tablet_bootstrap.cc:654] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.857851 4384 tablet_bootstrap.cc:654] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.859195 4391 tablet_bootstrap.cc:492] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:35.859298 4391 ts_tablet_manager.cc:1367] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:00:35.859758 4391 raft_consensus.cc:348] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:35.859898 4391 raft_consensus.cc:374] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.859939 4391 raft_consensus.cc:729] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.860069 4391 consensus_queue.cc:260] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:35.860352 4391 ts_tablet_manager.cc:1398] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:35.861119 4384 tablet_bootstrap.cc:492] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:35.861198 4384 ts_tablet_manager.cc:1367] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:35.861639 4384 raft_consensus.cc:348] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.861742 4384 raft_consensus.cc:374] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.861783 4384 raft_consensus.cc:729] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.861902 4384 consensus_queue.cc:260] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.862265 4384 ts_tablet_manager.cc:1398] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:35.862757 4387 tablet_bootstrap.cc:492] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:35.862836 4387 ts_tablet_manager.cc:1367] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:35.863315 4387 raft_consensus.cc:348] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.863427 4387 raft_consensus.cc:374] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.863466 4387 raft_consensus.cc:729] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.863579 4387 consensus_queue.cc:260] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.864161 4387 ts_tablet_manager.cc:1398] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:35.865645 32730 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:35.865764 32730 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:35.868081 32730 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 8d7208bb09f64d4d9011775ac93d37d7
I20240502 14:00:35.868759 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7208bb09f64d4d9011775ac93d37d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:35 UTC) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:35.868803 386 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7208bb09f64d4d9011775ac93d37d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:35 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:35.869060 4400 tablet_replica.cc:330] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:35.869041 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7208bb09f64d4d9011775ac93d37d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:35 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:35.869192 4400 raft_consensus.cc:2229] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:35.869354 4402 tablet_replica.cc:330] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:35.869346 4400 raft_consensus.cc:2258] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:35.869575 4402 raft_consensus.cc:2229] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:35.869820 4402 raft_consensus.cc:2258] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:35.869927 4400 ts_tablet_manager.cc:1875] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:35.870332 4402 ts_tablet_manager.cc:1875] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:35.872251 4400 ts_tablet_manager.cc:1888] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:35.872331 4400 log.cc:1196] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/8d7208bb09f64d4d9011775ac93d37d7
I20240502 14:00:35.872588 4402 ts_tablet_manager.cc:1888] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:35.872620 4400 ts_tablet_manager.cc:1909] T 8d7208bb09f64d4d9011775ac93d37d7 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting consensus metadata
I20240502 14:00:35.872666 4402 log.cc:1196] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/8d7208bb09f64d4d9011775ac93d37d7
I20240502 14:00:35.872936 4402 ts_tablet_manager.cc:1909] T 8d7208bb09f64d4d9011775ac93d37d7 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:35.873226 4401 tablet_replica.cc:330] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:35.873289 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 8d7208bb09f64d4d9011775ac93d37d7 (table rebalancer_dynamic_table_new_name [id=bbd44093088f4096b3f5e67abfaa184c]) successfully deleted
I20240502 14:00:35.873387 4401 raft_consensus.cc:2229] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:35.873530 4401 raft_consensus.cc:2258] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:35.873754 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 8d7208bb09f64d4d9011775ac93d37d7 (table rebalancer_dynamic_table_new_name [id=bbd44093088f4096b3f5e67abfaa184c]) successfully deleted
I20240502 14:00:35.874089 4401 ts_tablet_manager.cc:1875] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:35.876065 4401 ts_tablet_manager.cc:1888] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:35.876179 4401 log.cc:1196] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/8d7208bb09f64d4d9011775ac93d37d7
I20240502 14:00:35.876528 4401 ts_tablet_manager.cc:1909] T 8d7208bb09f64d4d9011775ac93d37d7 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:35.877115 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 8d7208bb09f64d4d9011775ac93d37d7 (table rebalancer_dynamic_table_new_name [id=bbd44093088f4096b3f5e67abfaa184c]) successfully deleted
I20240502 14:00:35.908746 4307 raft_consensus.cc:482] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:35.908893 4307 raft_consensus.cc:504] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.909334 4307 leader_election.cc:288] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:35.909621 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618c1b9af9504c03bd250d88c3a2b0ba" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:35.909653 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618c1b9af9504c03bd250d88c3a2b0ba" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:35.909771 2884 raft_consensus.cc:2454] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:35.909797 3283 raft_consensus.cc:2454] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:35.910043 609 leader_election.cc:302] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:35.910259 4307 raft_consensus.cc:2781] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:35.910310 4307 raft_consensus.cc:482] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:35.910353 4307 raft_consensus.cc:3037] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.911413 4307 raft_consensus.cc:504] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.911746 4307 leader_election.cc:288] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:35.912092 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618c1b9af9504c03bd250d88c3a2b0ba" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:35.912209 2884 raft_consensus.cc:3037] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.912387 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "618c1b9af9504c03bd250d88c3a2b0ba" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:35.912518 3283 raft_consensus.cc:3037] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.913177 2884 raft_consensus.cc:2454] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:35.913439 3283 raft_consensus.cc:2454] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:35.913669 609 leader_election.cc:302] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:35.914063 4307 raft_consensus.cc:2781] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:35.914198 4307 raft_consensus.cc:686] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:35.914342 4307 consensus_queue.cc:237] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.916368 32732 catalog_manager.cc:5484] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:35.930855 4274 raft_consensus.cc:482] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:35.930958 4274 raft_consensus.cc:504] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:35.931327 4274 leader_election.cc:288] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:35.931596 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0199aa852916433dafaf8a08720ca6d3" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:35.931742 3019 raft_consensus.cc:2454] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:35.931742 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0199aa852916433dafaf8a08720ca6d3" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:35.931896 805 raft_consensus.cc:2454] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:35.932076 3216 leader_election.cc:302] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:35.932235 4274 raft_consensus.cc:2781] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:35.932282 4274 raft_consensus.cc:482] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:35.932332 4274 raft_consensus.cc:3037] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.933214 4274 raft_consensus.cc:504] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:35.933506 4274 leader_election.cc:288] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:35.933851 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0199aa852916433dafaf8a08720ca6d3" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:35.933851 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0199aa852916433dafaf8a08720ca6d3" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:35.933990 805 raft_consensus.cc:3037] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.933990 3019 raft_consensus.cc:3037] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.934974 3019 raft_consensus.cc:2454] T 0199aa852916433dafaf8a08720ca6d3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:35.934978 805 raft_consensus.cc:2454] T 0199aa852916433dafaf8a08720ca6d3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:35.935246 3219 leader_election.cc:302] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:35.935426 4274 raft_consensus.cc:2781] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:35.935518 4274 raft_consensus.cc:686] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:35.935653 4274 consensus_queue.cc:237] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:35.937433 32732 catalog_manager.cc:5484] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:35.963934 4307 raft_consensus.cc:482] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:35.964092 4307 raft_consensus.cc:504] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.964581 4307 leader_election.cc:288] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:35.964890 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e843cd7c984ebfb532fec67f97e249" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:35.964913 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e843cd7c984ebfb532fec67f97e249" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:35.965041 2884 raft_consensus.cc:2454] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:35.965065 3283 raft_consensus.cc:2454] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:35.965314 609 leader_election.cc:302] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:35.965535 4307 raft_consensus.cc:2781] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:35.965598 4307 raft_consensus.cc:482] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:35.965653 4307 raft_consensus.cc:3037] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.966499 4307 raft_consensus.cc:504] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.966846 4307 leader_election.cc:288] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:35.967129 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e843cd7c984ebfb532fec67f97e249" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:35.967155 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8e843cd7c984ebfb532fec67f97e249" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:35.967242 3283 raft_consensus.cc:3037] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.967273 2884 raft_consensus.cc:3037] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:35.968084 3283 raft_consensus.cc:2454] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:35.968240 2884 raft_consensus.cc:2454] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:35.968364 607 leader_election.cc:302] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:35.968557 4307 raft_consensus.cc:2781] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:35.968672 4307 raft_consensus.cc:686] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:35.969008 4307 consensus_queue.cc:237] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:35.970661 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:35.971190 32730 catalog_manager.cc:5484] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:35.976526 3130 tablet_service.cc:1467] Processing CreateTablet for tablet da2e9b4563e74ca5b3c976ee73ec7afe (DEFAULT_TABLE table=rebalancer_dynamic_table [id=45b724ba067b44158e9a8a8f00adc9a3]), partition=
I20240502 14:00:35.976536 386 tablet_service.cc:1467] Processing CreateTablet for tablet da2e9b4563e74ca5b3c976ee73ec7afe (DEFAULT_TABLE table=rebalancer_dynamic_table [id=45b724ba067b44158e9a8a8f00adc9a3]), partition=
I20240502 14:00:35.976795 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2e9b4563e74ca5b3c976ee73ec7afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.976799 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2e9b4563e74ca5b3c976ee73ec7afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.978799 4409 tablet_bootstrap.cc:492] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:35.978790 2998 tablet_service.cc:1467] Processing CreateTablet for tablet da2e9b4563e74ca5b3c976ee73ec7afe (DEFAULT_TABLE table=rebalancer_dynamic_table [id=45b724ba067b44158e9a8a8f00adc9a3]), partition=
I20240502 14:00:35.979059 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2e9b4563e74ca5b3c976ee73ec7afe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.979214 4408 tablet_bootstrap.cc:492] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:35.979707 4409 tablet_bootstrap.cc:654] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.980163 4408 tablet_bootstrap.cc:654] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.981281 4386 tablet_bootstrap.cc:492] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:35.981994 4408 tablet_bootstrap.cc:492] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:35.982084 4408 ts_tablet_manager.cc:1367] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:35.982164 4409 tablet_bootstrap.cc:492] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:35.982259 4386 tablet_bootstrap.cc:654] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:35.982254 4409 ts_tablet_manager.cc:1367] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:35.982725 4409 raft_consensus.cc:348] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:35.982867 4409 raft_consensus.cc:374] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.982909 4409 raft_consensus.cc:729] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.982933 4408 raft_consensus.cc:348] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:35.983055 4409 consensus_queue.cc:260] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:35.983306 4408 raft_consensus.cc:374] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.983407 4409 ts_tablet_manager.cc:1398] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:35.983397 4408 raft_consensus.cc:729] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.983639 4386 tablet_bootstrap.cc:492] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:35.983551 4408 consensus_queue.cc:260] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:35.983739 4386 ts_tablet_manager.cc:1367] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:35.983991 4408 ts_tablet_manager.cc:1398] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:35.984192 4386 raft_consensus.cc:348] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:35.984309 4386 raft_consensus.cc:374] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:35.984334 4386 raft_consensus.cc:729] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:35.984445 4386 consensus_queue.cc:260] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:35.984700 4386 ts_tablet_manager.cc:1398] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:36.046875 4194 raft_consensus.cc:482] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:36.047012 4194 raft_consensus.cc:504] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.047405 4194 leader_election.cc:288] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:36.047744 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2e9b4563e74ca5b3c976ee73ec7afe" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:36.047889 3019 raft_consensus.cc:2454] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:36.047955 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2e9b4563e74ca5b3c976ee73ec7afe" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:36.048105 405 raft_consensus.cc:2454] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:36.048131 3084 leader_election.cc:302] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:36.048318 4194 raft_consensus.cc:2781] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:36.048372 4194 raft_consensus.cc:482] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:36.048417 4194 raft_consensus.cc:3037] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.049232 4194 raft_consensus.cc:504] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.049574 4194 leader_election.cc:288] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:36.049896 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2e9b4563e74ca5b3c976ee73ec7afe" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:36.049963 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2e9b4563e74ca5b3c976ee73ec7afe" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:36.050024 405 raft_consensus.cc:3037] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.050077 3019 raft_consensus.cc:3037] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.050799 3019 raft_consensus.cc:2454] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:36.050809 405 raft_consensus.cc:2454] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:36.051079 3086 leader_election.cc:302] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:36.051235 4194 raft_consensus.cc:2781] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:36.051321 4194 raft_consensus.cc:686] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:36.051445 4194 consensus_queue.cc:237] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.053241 32726 catalog_manager.cc:5484] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:36.157410 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:36.161244 3019 raft_consensus.cc:1261] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:36.161284 405 raft_consensus.cc:1261] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:36.161712 4194 consensus_queue.cc:1035] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.161870 4362 consensus_queue.cc:1035] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.164981 4417 tablet.cc:1722] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:36.165302 4417 tablet.cc:1620] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:36.166018 32726 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet da2e9b4563e74ca5b3c976ee73ec7afe (table rebalancer_dynamic_table [id=45b724ba067b44158e9a8a8f00adc9a3]). Expected version 1 got 0
I20240502 14:00:36.167117 32732 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet da2e9b4563e74ca5b3c976ee73ec7afe (table rebalancer_dynamic_table [id=45b724ba067b44158e9a8a8f00adc9a3]). Expected version 1 got 0
I20240502 14:00:36.167287 4419 tablet.cc:1722] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:36.167627 4419 tablet.cc:1620] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:36.167870 4374 tablet.cc:1722] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:36.168167 4374 tablet.cc:1620] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:36.169744 32732 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet da2e9b4563e74ca5b3c976ee73ec7afe (table rebalancer_dynamic_table [id=45b724ba067b44158e9a8a8f00adc9a3]). Expected version 1 got 0
I20240502 14:00:36.171943 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=45b724ba067b44158e9a8a8f00adc9a3] alter complete (version 1)
I20240502 14:00:36.275524 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:36.281715 4417 tablet.cc:1722] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:36.282033 4374 tablet.cc:1722] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:36.282241 4417 tablet.cc:1620] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:36.282301 4374 tablet.cc:1620] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:36.282332 4419 tablet.cc:1722] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:36.282636 4419 tablet.cc:1620] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:36.288609 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=45b724ba067b44158e9a8a8f00adc9a3] alter complete (version 2)
I20240502 14:00:36.300721 4359 consensus_queue.cc:1035] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.304255 4403 consensus_queue.cc:1035] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.334241 4370 consensus_queue.cc:1035] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.337345 4370 consensus_queue.cc:1035] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.392494 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:36.397967 4417 tablet.cc:1722] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:36.398875 4419 tablet.cc:1722] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:36.401362 4374 tablet.cc:1722] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:36.405234 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=45b724ba067b44158e9a8a8f00adc9a3] alter complete (version 3)
I20240502 14:00:36.472446 4403 consensus_queue.cc:1035] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.475464 4403 consensus_queue.cc:1035] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.501664 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:36.508107 2998 tablet_service.cc:1467] Processing CreateTablet for tablet c1dc8816aa514d1abcce63ce0e0bf357 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=409fb32bfca44ecaa3f9d301ac14ce3a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:36.508147 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:36.508347 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:36.508458 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1dc8816aa514d1abcce63ce0e0bf357. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.509073 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 184c03808a1e470dba9c6bfa8e04774a (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=409fb32bfca44ecaa3f9d301ac14ce3a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:36.509337 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 184c03808a1e470dba9c6bfa8e04774a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.510782 4429 tablet_bootstrap.cc:492] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:36.510987 3264 tablet_service.cc:1467] Processing CreateTablet for tablet a37ad1cd10564bcebe788fb8c49a66a8 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=409fb32bfca44ecaa3f9d301ac14ce3a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:36.511241 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a37ad1cd10564bcebe788fb8c49a66a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.511873 4429 tablet_bootstrap.cc:654] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.512409 4430 tablet_bootstrap.cc:492] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:36.512777 3130 tablet_service.cc:1467] Processing CreateTablet for tablet c1dc8816aa514d1abcce63ce0e0bf357 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=409fb32bfca44ecaa3f9d301ac14ce3a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:36.513087 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1dc8816aa514d1abcce63ce0e0bf357. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.513682 386 tablet_service.cc:1467] Processing CreateTablet for tablet c1dc8816aa514d1abcce63ce0e0bf357 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=409fb32bfca44ecaa3f9d301ac14ce3a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:36.513814 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet da2e9b4563e74ca5b3c976ee73ec7afe
I20240502 14:00:36.513971 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1dc8816aa514d1abcce63ce0e0bf357. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.514315 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 184c03808a1e470dba9c6bfa8e04774a (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=409fb32bfca44ecaa3f9d301ac14ce3a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:36.514398 4430 tablet_bootstrap.cc:654] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.514570 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 184c03808a1e470dba9c6bfa8e04774a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.515336 4432 tablet_bootstrap.cc:492] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:36.516304 4432 tablet_bootstrap.cc:654] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.516443 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet da2e9b4563e74ca5b3c976ee73ec7afe with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:36 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:36.517102 388 tablet_service.cc:1514] Processing DeleteTablet for tablet da2e9b4563e74ca5b3c976ee73ec7afe with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:36 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:36.517774 4437 tablet_bootstrap.cc:492] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:36.517982 653 tablet_service.cc:1467] Processing CreateTablet for tablet a37ad1cd10564bcebe788fb8c49a66a8 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=409fb32bfca44ecaa3f9d301ac14ce3a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:36.518085 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet da2e9b4563e74ca5b3c976ee73ec7afe with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:36 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:36.518191 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 184c03808a1e470dba9c6bfa8e04774a (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=409fb32bfca44ecaa3f9d301ac14ce3a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:36.518271 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a37ad1cd10564bcebe788fb8c49a66a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.518436 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 184c03808a1e470dba9c6bfa8e04774a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.518442 4440 tablet_replica.cc:330] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:36.518621 4440 raft_consensus.cc:2229] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:36.518736 4440 raft_consensus.cc:2258] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:36.519814 4434 tablet_bootstrap.cc:492] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:36.520826 4434 tablet_bootstrap.cc:654] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.521235 4429 tablet_bootstrap.cc:492] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:36.521193 2998 tablet_service.cc:1467] Processing CreateTablet for tablet a37ad1cd10564bcebe788fb8c49a66a8 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=409fb32bfca44ecaa3f9d301ac14ce3a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:36.521330 4429 ts_tablet_manager.cc:1367] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:00:36.521387 4437 tablet_bootstrap.cc:654] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.521437 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a37ad1cd10564bcebe788fb8c49a66a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.521806 4429 raft_consensus.cc:348] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:36.521919 4429 raft_consensus.cc:374] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.521965 4429 raft_consensus.cc:729] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.522089 4429 consensus_queue.cc:260] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:36.522411 4429 ts_tablet_manager.cc:1398] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:36.522586 4429 tablet_bootstrap.cc:492] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:36.523087 4430 tablet_bootstrap.cc:492] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:36.523173 4430 ts_tablet_manager.cc:1367] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:00:36.523408 4434 tablet_bootstrap.cc:492] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:36.523464 4429 tablet_bootstrap.cc:654] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.523507 4434 ts_tablet_manager.cc:1367] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:36.523587 4438 tablet_replica.cc:330] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:36.523707 4438 raft_consensus.cc:2229] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:36.523694 4430 raft_consensus.cc:348] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:36.523805 4430 raft_consensus.cc:374] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.523849 4438 raft_consensus.cc:2258] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:36.523871 4430 raft_consensus.cc:729] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.523996 4430 consensus_queue.cc:260] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:36.523985 4434 raft_consensus.cc:348] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:36.524080 4433 tablet_replica.cc:330] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:36.524082 4434 raft_consensus.cc:374] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.524116 4434 raft_consensus.cc:729] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.524212 4433 raft_consensus.cc:2229] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:36.524253 4430 ts_tablet_manager.cc:1398] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:36.524233 4434 consensus_queue.cc:260] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:36.524415 4430 tablet_bootstrap.cc:492] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:36.524518 4434 ts_tablet_manager.cc:1398] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:36.525354 4430 tablet_bootstrap.cc:654] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.526479 4432 tablet_bootstrap.cc:492] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:36.526583 4432 ts_tablet_manager.cc:1367] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:00:36.527192 4429 tablet_bootstrap.cc:492] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:36.527273 4429 ts_tablet_manager.cc:1367] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:36.527192 4432 raft_consensus.cc:348] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:36.527334 4432 raft_consensus.cc:374] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.527374 4432 raft_consensus.cc:729] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.527462 4433 raft_consensus.cc:2258] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:36.527513 4432 consensus_queue.cc:260] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:36.527738 4437 tablet_bootstrap.cc:492] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:36.527709 4429 raft_consensus.cc:348] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:36.527783 4432 ts_tablet_manager.cc:1398] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:36.527796 4429 raft_consensus.cc:374] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.527874 4437 ts_tablet_manager.cc:1367] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:36.528087 4433 ts_tablet_manager.cc:1875] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:36.528321 4437 raft_consensus.cc:348] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:36.528419 4437 raft_consensus.cc:374] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.528445 4437 raft_consensus.cc:729] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.527865 4429 raft_consensus.cc:729] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.528584 4437 consensus_queue.cc:260] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:36.528723 4429 consensus_queue.cc:260] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:36.528851 4437 ts_tablet_manager.cc:1398] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:36.529206 4429 ts_tablet_manager.cc:1398] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:36.530107 4430 tablet_bootstrap.cc:492] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:36.530208 4430 ts_tablet_manager.cc:1367] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 14:00:36.530278 4433 ts_tablet_manager.cc:1888] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:36.530357 4433 log.cc:1196] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/da2e9b4563e74ca5b3c976ee73ec7afe
I20240502 14:00:36.530611 4430 raft_consensus.cc:348] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.530655 4433 ts_tablet_manager.cc:1909] T da2e9b4563e74ca5b3c976ee73ec7afe P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:36.530702 4430 raft_consensus.cc:374] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.530737 4430 raft_consensus.cc:729] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.530881 4430 consensus_queue.cc:260] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.531150 4430 ts_tablet_manager.cc:1398] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:36.531345 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet da2e9b4563e74ca5b3c976ee73ec7afe (table rebalancer_dynamic_table_new_name [id=45b724ba067b44158e9a8a8f00adc9a3]) successfully deleted
I20240502 14:00:36.531510 4438 ts_tablet_manager.cc:1875] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:36.531607 4440 ts_tablet_manager.cc:1875] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:36.533906 4438 ts_tablet_manager.cc:1888] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:36.534065 4438 log.cc:1196] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/da2e9b4563e74ca5b3c976ee73ec7afe
I20240502 14:00:36.534448 4438 ts_tablet_manager.cc:1909] T da2e9b4563e74ca5b3c976ee73ec7afe P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:36.535223 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet da2e9b4563e74ca5b3c976ee73ec7afe (table rebalancer_dynamic_table_new_name [id=45b724ba067b44158e9a8a8f00adc9a3]) successfully deleted
I20240502 14:00:36.535295 4440 ts_tablet_manager.cc:1888] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:36.535358 4440 log.cc:1196] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/da2e9b4563e74ca5b3c976ee73ec7afe
I20240502 14:00:36.535720 4440 ts_tablet_manager.cc:1909] T da2e9b4563e74ca5b3c976ee73ec7afe P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:36.536424 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet da2e9b4563e74ca5b3c976ee73ec7afe (table rebalancer_dynamic_table_new_name [id=45b724ba067b44158e9a8a8f00adc9a3]) successfully deleted
I20240502 14:00:36.536715 4441 tablet_bootstrap.cc:492] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:36.537120 4429 tablet_bootstrap.cc:492] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:36.538048 4340 raft_consensus.cc:482] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:36.538138 4340 raft_consensus.cc:504] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:36.538467 4340 leader_election.cc:288] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:36.538929 4441 tablet_bootstrap.cc:654] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.538969 4429 tablet_bootstrap.cc:654] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.539098 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1dc8816aa514d1abcce63ce0e0bf357" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:36.539227 3019 raft_consensus.cc:2454] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:36.539772 341 leader_election.cc:302] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:36.539968 4340 raft_consensus.cc:2781] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:36.539948 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1dc8816aa514d1abcce63ce0e0bf357" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:36.540014 4340 raft_consensus.cc:482] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:36.540050 4340 raft_consensus.cc:3037] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.540076 3150 raft_consensus.cc:2454] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:36.540625 4441 tablet_bootstrap.cc:492] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:36.540707 4441 ts_tablet_manager.cc:1367] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:36.541060 4340 raft_consensus.cc:504] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:36.541217 4441 raft_consensus.cc:348] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.541311 4441 raft_consensus.cc:374] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.541342 4441 raft_consensus.cc:729] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.541435 4340 leader_election.cc:288] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:36.541478 4441 consensus_queue.cc:260] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.541692 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1dc8816aa514d1abcce63ce0e0bf357" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:36.541782 3019 raft_consensus.cc:3037] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.541793 4441 ts_tablet_manager.cc:1398] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:36.542457 3019 raft_consensus.cc:2454] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:36.543429 341 leader_election.cc:302] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:36.543721 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1dc8816aa514d1abcce63ce0e0bf357" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:36.543864 3150 raft_consensus.cc:3037] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.544883 3150 raft_consensus.cc:2454] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:36.545109 4340 raft_consensus.cc:2781] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:36.545197 4340 raft_consensus.cc:686] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:36.545248 4326 raft_consensus.cc:482] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:36.545325 4340 consensus_queue.cc:237] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:36.545333 4326 raft_consensus.cc:504] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:36.545725 4326 leader_election.cc:288] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:36.546098 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "184c03808a1e470dba9c6bfa8e04774a" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:36.546239 3283 raft_consensus.cc:2454] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:36.546826 2820 leader_election.cc:302] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:36.547001 4326 raft_consensus.cc:2781] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:36.546998 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "184c03808a1e470dba9c6bfa8e04774a" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:36.547061 4326 raft_consensus.cc:482] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:36.547101 4326 raft_consensus.cc:3037] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.547123 3019 raft_consensus.cc:2454] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:36.547915 32732 catalog_manager.cc:5484] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:36.548076 4429 tablet_bootstrap.cc:492] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:36.548161 4429 ts_tablet_manager.cc:1367] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:00:36.548197 4326 raft_consensus.cc:504] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:36.548545 4326 leader_election.cc:288] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:36.548606 4429 raft_consensus.cc:348] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.548702 4429 raft_consensus.cc:374] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.548738 4429 raft_consensus.cc:729] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.548794 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "184c03808a1e470dba9c6bfa8e04774a" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:36.548858 4429 consensus_queue.cc:260] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.548894 3019 raft_consensus.cc:3037] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.549144 4429 ts_tablet_manager.cc:1398] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:36.549481 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "184c03808a1e470dba9c6bfa8e04774a" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:36.549593 3283 raft_consensus.cc:3037] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.549844 3019 raft_consensus.cc:2454] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:36.550150 2819 leader_election.cc:302] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:36.550297 4326 raft_consensus.cc:2781] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:36.550366 4326 raft_consensus.cc:686] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:36.550518 3283 raft_consensus.cc:2454] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:36.550487 4326 consensus_queue.cc:237] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:36.552950 32732 catalog_manager.cc:5484] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:36.616664 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:36.621755 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb84b6c3f0d4d188b25084a72e48227 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d7294ba0897749a6b8e2abd277f0a904]), partition=
I20240502 14:00:36.621750 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb84b6c3f0d4d188b25084a72e48227 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d7294ba0897749a6b8e2abd277f0a904]), partition=
I20240502 14:00:36.622064 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb84b6c3f0d4d188b25084a72e48227. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.622031 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb84b6c3f0d4d188b25084a72e48227 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d7294ba0897749a6b8e2abd277f0a904]), partition=
I20240502 14:00:36.622071 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb84b6c3f0d4d188b25084a72e48227. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.622273 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb84b6c3f0d4d188b25084a72e48227. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.623791 4430 tablet_bootstrap.cc:492] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:36.624327 4432 tablet_bootstrap.cc:492] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:36.624589 4429 tablet_bootstrap.cc:492] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:36.625137 4430 tablet_bootstrap.cc:654] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.625211 4432 tablet_bootstrap.cc:654] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.625519 4429 tablet_bootstrap.cc:654] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:36.626363 4432 tablet_bootstrap.cc:492] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:36.626456 4432 ts_tablet_manager.cc:1367] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:36.626557 4430 tablet_bootstrap.cc:492] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:36.626657 4430 ts_tablet_manager.cc:1367] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:36.626873 4429 tablet_bootstrap.cc:492] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:36.626878 4432 raft_consensus.cc:348] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:36.626964 4429 ts_tablet_manager.cc:1367] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:36.626983 4432 raft_consensus.cc:374] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.627024 4432 raft_consensus.cc:729] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.627152 4432 consensus_queue.cc:260] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:36.627431 4432 ts_tablet_manager.cc:1398] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:36.627429 4429 raft_consensus.cc:348] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:36.627555 4429 raft_consensus.cc:374] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.627593 4429 raft_consensus.cc:729] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.627137 4430 raft_consensus.cc:348] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:36.627647 4430 raft_consensus.cc:374] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:36.627691 4430 raft_consensus.cc:729] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:36.627743 4429 consensus_queue.cc:260] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:36.627810 4430 consensus_queue.cc:260] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:36.628095 4430 ts_tablet_manager.cc:1398] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 14:00:36.628702 4429 ts_tablet_manager.cc:1398] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:36.651942 4139 raft_consensus.cc:482] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:36.652072 4139 raft_consensus.cc:504] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.652475 4139 leader_election.cc:288] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:36.652791 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a37ad1cd10564bcebe788fb8c49a66a8" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:36.652819 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a37ad1cd10564bcebe788fb8c49a66a8" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:36.652923 3283 raft_consensus.cc:2454] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:36.652962 672 raft_consensus.cc:2454] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:36.653190 2952 leader_election.cc:302] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:36.653355 4139 raft_consensus.cc:2781] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:36.653416 4139 raft_consensus.cc:482] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:36.653443 4139 raft_consensus.cc:3037] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.654346 4139 raft_consensus.cc:504] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.654672 4139 leader_election.cc:288] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:36.654976 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a37ad1cd10564bcebe788fb8c49a66a8" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:36.655020 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a37ad1cd10564bcebe788fb8c49a66a8" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:36.655107 3283 raft_consensus.cc:3037] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.655154 672 raft_consensus.cc:3037] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.656028 672 raft_consensus.cc:2454] T a37ad1cd10564bcebe788fb8c49a66a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:36.656088 3283 raft_consensus.cc:2454] T a37ad1cd10564bcebe788fb8c49a66a8 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:36.656329 2951 leader_election.cc:302] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:36.656589 4139 raft_consensus.cc:2781] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:36.656675 4139 raft_consensus.cc:686] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:36.656802 4139 consensus_queue.cc:237] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:36.658993 32731 catalog_manager.cc:5484] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:36.725919 4274 raft_consensus.cc:482] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:36.726042 4274 raft_consensus.cc:504] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:36.726419 4274 leader_election.cc:288] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:36.726737 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb84b6c3f0d4d188b25084a72e48227" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:36.726778 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb84b6c3f0d4d188b25084a72e48227" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:36.726869 3019 raft_consensus.cc:2454] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:36.726935 3150 raft_consensus.cc:2454] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:36.727141 3216 leader_election.cc:302] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:36.727306 4274 raft_consensus.cc:2781] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:36.727365 4274 raft_consensus.cc:482] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:36.727392 4274 raft_consensus.cc:3037] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.728353 4274 raft_consensus.cc:504] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:36.728734 4274 leader_election.cc:288] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:36.729032 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb84b6c3f0d4d188b25084a72e48227" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:36.729143 3019 raft_consensus.cc:3037] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.729198 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb84b6c3f0d4d188b25084a72e48227" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:36.729326 3150 raft_consensus.cc:3037] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:36.729995 3019 raft_consensus.cc:2454] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:36.730273 3150 raft_consensus.cc:2454] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:36.730378 3216 leader_election.cc:302] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:36.730556 4274 raft_consensus.cc:2781] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:36.730646 4274 raft_consensus.cc:686] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:36.730782 4274 consensus_queue.cc:237] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:36.732682 32731 catalog_manager.cc:5484] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:36.850965 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:36.854969 3150 raft_consensus.cc:1261] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:36.855028 3019 raft_consensus.cc:1261] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:36.855387 4274 consensus_queue.cc:1035] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.855835 4370 consensus_queue.cc:1035] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.858390 32732 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 8fb84b6c3f0d4d188b25084a72e48227 (table rebalancer_dynamic_table [id=d7294ba0897749a6b8e2abd277f0a904]). Expected version 1 got 0
I20240502 14:00:36.859506 32732 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet 8fb84b6c3f0d4d188b25084a72e48227 (table rebalancer_dynamic_table [id=d7294ba0897749a6b8e2abd277f0a904]). Expected version 1 got 0
I20240502 14:00:36.861395 4458 tablet.cc:1722] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:36.861542 4419 tablet.cc:1722] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:36.861789 4458 tablet.cc:1620] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:36.861843 4419 tablet.cc:1620] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:36.861841 32732 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 8fb84b6c3f0d4d188b25084a72e48227 (table rebalancer_dynamic_table [id=d7294ba0897749a6b8e2abd277f0a904]). Expected version 1 got 0
I20240502 14:00:36.863255 4417 tablet.cc:1722] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:36.863530 4417 tablet.cc:1620] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:36.866519 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=d7294ba0897749a6b8e2abd277f0a904] alter complete (version 1)
I20240502 14:00:36.921926 4340 consensus_queue.cc:1035] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.926637 4340 consensus_queue.cc:1035] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:36.969002 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:36.975111 4419 tablet.cc:1722] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:36.974634 4458 tablet.cc:1722] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:36.975479 4419 tablet.cc:1620] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:36.975651 4458 tablet.cc:1620] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:36.976631 4417 tablet.cc:1722] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:36.976927 4417 tablet.cc:1620] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:36.981806 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=d7294ba0897749a6b8e2abd277f0a904] alter complete (version 2)
I20240502 14:00:36.996912 4449 consensus_queue.cc:1035] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.000587 4326 consensus_queue.cc:1035] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.084825 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:37.090080 4458 tablet.cc:1722] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:37.090586 4419 tablet.cc:1722] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:37.090690 4417 tablet.cc:1722] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:37.096169 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=d7294ba0897749a6b8e2abd277f0a904] alter complete (version 3)
I20240502 14:00:37.165575 4468 consensus_queue.cc:1035] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.168720 4468 consensus_queue.cc:1035] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.172134 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:37.178712 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 907ccfaebc3c4181bda0d3f03aba050c (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=f203281184c04e388bc5589f549f4a64]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.178747 2998 tablet_service.cc:1467] Processing CreateTablet for tablet 907ccfaebc3c4181bda0d3f03aba050c (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=f203281184c04e388bc5589f549f4a64]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.179033 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 907ccfaebc3c4181bda0d3f03aba050c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.179033 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 907ccfaebc3c4181bda0d3f03aba050c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.180368 2997 tablet_service.cc:1467] Processing CreateTablet for tablet b985ba4b9f534827919cbb24daa4de0e (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=f203281184c04e388bc5589f549f4a64]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.180368 2999 tablet_service.cc:1467] Processing CreateTablet for tablet f808844727624864a0da274de95c2b05 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=f203281184c04e388bc5589f549f4a64]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.180712 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f808844727624864a0da274de95c2b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.180979 2866 tablet_service.cc:1467] Processing CreateTablet for tablet f808844727624864a0da274de95c2b05 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=f203281184c04e388bc5589f549f4a64]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.181232 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f808844727624864a0da274de95c2b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.181690 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 907ccfaebc3c4181bda0d3f03aba050c (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=f203281184c04e388bc5589f549f4a64]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.181985 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 907ccfaebc3c4181bda0d3f03aba050c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.182756 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b985ba4b9f534827919cbb24daa4de0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.183696 3132 tablet_service.cc:1467] Processing CreateTablet for tablet f808844727624864a0da274de95c2b05 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=f203281184c04e388bc5589f549f4a64]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.183971 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f808844727624864a0da274de95c2b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.184069 3263 tablet_service.cc:1467] Processing CreateTablet for tablet b985ba4b9f534827919cbb24daa4de0e (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=f203281184c04e388bc5589f549f4a64]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.184306 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b985ba4b9f534827919cbb24daa4de0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.184530 4472 tablet_bootstrap.cc:492] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:37.185081 4474 tablet_bootstrap.cc:492] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:37.185778 4472 tablet_bootstrap.cc:654] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.186141 4474 tablet_bootstrap.cc:654] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.186699 386 tablet_service.cc:1467] Processing CreateTablet for tablet b985ba4b9f534827919cbb24daa4de0e (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=f203281184c04e388bc5589f549f4a64]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.186791 4475 tablet_bootstrap.cc:492] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:37.187021 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b985ba4b9f534827919cbb24daa4de0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.187605 4475 tablet_bootstrap.cc:654] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.188125 4472 tablet_bootstrap.cc:492] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:37.188203 4472 ts_tablet_manager.cc:1367] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:37.188665 4472 raft_consensus.cc:348] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.188779 4472 raft_consensus.cc:374] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.188812 4472 raft_consensus.cc:729] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.188947 4472 consensus_queue.cc:260] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.189500 4479 tablet_bootstrap.cc:492] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:37.190351 4472 ts_tablet_manager.cc:1398] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:37.190559 4472 tablet_bootstrap.cc:492] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:37.190845 4479 tablet_bootstrap.cc:654] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.191854 4472 tablet_bootstrap.cc:654] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.192426 4473 tablet_bootstrap.cc:492] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:37.193483 4473 tablet_bootstrap.cc:654] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.193696 4479 tablet_bootstrap.cc:492] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:37.193799 4479 ts_tablet_manager.cc:1367] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:37.194643 4472 tablet_bootstrap.cc:492] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:37.194727 4472 ts_tablet_manager.cc:1367] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:37.194968 4474 tablet_bootstrap.cc:492] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:37.195065 4474 ts_tablet_manager.cc:1367] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:37.195075 4472 raft_consensus.cc:348] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.195164 4472 raft_consensus.cc:374] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.195185 4472 raft_consensus.cc:729] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.195291 4472 consensus_queue.cc:260] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.195565 4472 ts_tablet_manager.cc:1398] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.195695 4474 raft_consensus.cc:348] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.195804 4474 raft_consensus.cc:374] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.195880 4474 raft_consensus.cc:729] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.195864 4479 raft_consensus.cc:348] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.195967 4479 raft_consensus.cc:374] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.195999 4479 raft_consensus.cc:729] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.196027 4474 consensus_queue.cc:260] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.196107 4479 consensus_queue.cc:260] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.196314 4474 ts_tablet_manager.cc:1398] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 14:00:37.197088 4475 tablet_bootstrap.cc:492] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:37.197190 4475 ts_tablet_manager.cc:1367] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240502 14:00:37.197670 4475 raft_consensus.cc:348] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.197813 4475 raft_consensus.cc:374] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.197870 4475 raft_consensus.cc:729] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.198027 4475 consensus_queue.cc:260] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.199194 4479 ts_tablet_manager.cc:1398] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:37.199535 4475 ts_tablet_manager.cc:1398] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:37.199769 4475 tablet_bootstrap.cc:492] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:37.199851 4473 tablet_bootstrap.cc:492] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:37.200030 4473 ts_tablet_manager.cc:1367] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:37.200373 32732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:37.200512 32732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:37.200707 4473 raft_consensus.cc:348] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.200884 4473 raft_consensus.cc:374] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.200969 4473 raft_consensus.cc:729] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.201170 4473 consensus_queue.cc:260] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.201505 4473 ts_tablet_manager.cc:1398] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.202699 4473 tablet_bootstrap.cc:492] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:37.203153 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 8fb84b6c3f0d4d188b25084a72e48227
I20240502 14:00:37.204054 4475 tablet_bootstrap.cc:654] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.204279 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb84b6c3f0d4d188b25084a72e48227 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:37 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:37.204594 2997 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb84b6c3f0d4d188b25084a72e48227 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:37 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:37.204253 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb84b6c3f0d4d188b25084a72e48227 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:37 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:37.204876 4483 tablet_replica.cc:330] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:37.204543 4317 raft_consensus.cc:482] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:37.205029 4483 raft_consensus.cc:2229] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:37.205021 4317 raft_consensus.cc:504] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.205147 4483 raft_consensus.cc:2258] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:37.205430 4317 leader_election.cc:288] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:37.205749 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b985ba4b9f534827919cbb24daa4de0e" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:37.205811 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b985ba4b9f534827919cbb24daa4de0e" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:37.205916 409 raft_consensus.cc:2454] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:37.205984 4483 ts_tablet_manager.cc:1875] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
W20240502 14:00:37.206269 3216 leader_election.cc:341] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:00:37.206393 3218 leader_election.cc:302] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters: 452ecbc085e54db49c3b01c20a96f860
I20240502 14:00:37.206559 4317 raft_consensus.cc:2781] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:37.206606 4317 raft_consensus.cc:482] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:37.206653 4317 raft_consensus.cc:3037] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.207232 4473 tablet_bootstrap.cc:654] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.207733 4317 raft_consensus.cc:504] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.207966 4483 ts_tablet_manager.cc:1888] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:37.208083 4483 log.cc:1196] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/8fb84b6c3f0d4d188b25084a72e48227
I20240502 14:00:37.208137 4317 leader_election.cc:288] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:37.208480 4483 ts_tablet_manager.cc:1909] T 8fb84b6c3f0d4d188b25084a72e48227 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:37.208446 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b985ba4b9f534827919cbb24daa4de0e" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
W20240502 14:00:37.208834 3216 leader_election.cc:341] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:00:37.208909 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b985ba4b9f534827919cbb24daa4de0e" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:37.209028 409 raft_consensus.cc:3037] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.209175 4484 tablet_replica.cc:330] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:37.209242 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 8fb84b6c3f0d4d188b25084a72e48227 (table rebalancer_dynamic_table_new_name [id=d7294ba0897749a6b8e2abd277f0a904]) successfully deleted
I20240502 14:00:37.209477 4484 raft_consensus.cc:2229] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:37.209619 4484 raft_consensus.cc:2258] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:37.210074 409 raft_consensus.cc:2454] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:37.210276 4484 ts_tablet_manager.cc:1875] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:37.210323 4475 tablet_bootstrap.cc:492] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:37.210399 4475 ts_tablet_manager.cc:1367] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 14:00:37.210888 4475 raft_consensus.cc:348] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.211009 4475 raft_consensus.cc:374] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.211055 4475 raft_consensus.cc:729] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.211014 3218 leader_election.cc:302] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters: 452ecbc085e54db49c3b01c20a96f860
I20240502 14:00:37.211154 4475 consensus_queue.cc:260] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.211455 4370 raft_consensus.cc:2781] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:37.211539 4370 raft_consensus.cc:686] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:37.211647 4370 consensus_queue.cc:237] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.211463 4475 ts_tablet_manager.cc:1398] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.212381 4473 tablet_bootstrap.cc:492] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:37.212448 4484 ts_tablet_manager.cc:1888] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:37.212471 4473 ts_tablet_manager.cc:1367] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:37.212531 4484 log.cc:1196] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/8fb84b6c3f0d4d188b25084a72e48227
I20240502 14:00:37.212605 4485 tablet_replica.cc:330] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:37.212718 4485 raft_consensus.cc:2229] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:37.212806 4484 ts_tablet_manager.cc:1909] T 8fb84b6c3f0d4d188b25084a72e48227 P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:37.212913 4485 raft_consensus.cc:2258] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:37.213048 4473 raft_consensus.cc:348] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.213130 4473 raft_consensus.cc:374] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.213150 4473 raft_consensus.cc:729] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.213255 4473 consensus_queue.cc:260] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.213415 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 8fb84b6c3f0d4d188b25084a72e48227 (table rebalancer_dynamic_table_new_name [id=d7294ba0897749a6b8e2abd277f0a904]) successfully deleted
I20240502 14:00:37.213689 4485 ts_tablet_manager.cc:1875] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:37.214177 4473 ts_tablet_manager.cc:1398] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:37.214404 4473 tablet_bootstrap.cc:492] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:37.214416 32726 catalog_manager.cc:5484] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:37.216068 4485 ts_tablet_manager.cc:1888] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:37.216213 4473 tablet_bootstrap.cc:654] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.216958 4485 log.cc:1196] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/8fb84b6c3f0d4d188b25084a72e48227
I20240502 14:00:37.217325 4485 ts_tablet_manager.cc:1909] T 8fb84b6c3f0d4d188b25084a72e48227 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:37.217365 4473 tablet_bootstrap.cc:492] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:37.217453 4473 ts_tablet_manager.cc:1367] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:37.217864 4473 raft_consensus.cc:348] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.217979 4473 raft_consensus.cc:374] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.218015 4473 raft_consensus.cc:729] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.218087 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 8fb84b6c3f0d4d188b25084a72e48227 (table rebalancer_dynamic_table_new_name [id=d7294ba0897749a6b8e2abd277f0a904]) successfully deleted
I20240502 14:00:37.218159 4473 consensus_queue.cc:260] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.218439 4473 ts_tablet_manager.cc:1398] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.305799 32726 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:37.311476 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 529244fd7ea14a60978d3f8fa6ad6efd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c3e4d6b3e7414735b6367d0e3bd2d3e6]), partition=
I20240502 14:00:37.311780 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 529244fd7ea14a60978d3f8fa6ad6efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.311862 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 529244fd7ea14a60978d3f8fa6ad6efd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c3e4d6b3e7414735b6367d0e3bd2d3e6]), partition=
I20240502 14:00:37.312011 4370 raft_consensus.cc:482] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:37.312152 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 529244fd7ea14a60978d3f8fa6ad6efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.312111 4370 raft_consensus.cc:504] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.312496 4370 leader_election.cc:288] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:37.314013 4473 tablet_bootstrap.cc:492] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:37.314224 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 529244fd7ea14a60978d3f8fa6ad6efd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c3e4d6b3e7414735b6367d0e3bd2d3e6]), partition=
I20240502 14:00:37.314388 4475 tablet_bootstrap.cc:492] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:37.314460 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 529244fd7ea14a60978d3f8fa6ad6efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.315037 4473 tablet_bootstrap.cc:654] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.315295 4475 tablet_bootstrap.cc:654] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.316255 4473 tablet_bootstrap.cc:492] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:37.316350 4473 ts_tablet_manager.cc:1367] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:37.316738 4472 tablet_bootstrap.cc:492] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:37.316787 4475 tablet_bootstrap.cc:492] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:37.316864 4475 ts_tablet_manager.cc:1367] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:37.316867 4473 raft_consensus.cc:348] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.316982 4473 raft_consensus.cc:374] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.317016 4473 raft_consensus.cc:729] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.317142 4473 consensus_queue.cc:260] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.317305 4475 raft_consensus.cc:348] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.317396 4475 raft_consensus.cc:374] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.317427 4475 raft_consensus.cc:729] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.317445 4473 ts_tablet_manager.cc:1398] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.317548 4472 tablet_bootstrap.cc:654] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.317554 4475 consensus_queue.cc:260] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.317826 4475 ts_tablet_manager.cc:1398] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.318305 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "907ccfaebc3c4181bda0d3f03aba050c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:37.318389 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "907ccfaebc3c4181bda0d3f03aba050c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:37.318418 3019 raft_consensus.cc:2454] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:37.318490 3150 raft_consensus.cc:2454] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:37.318722 3216 leader_election.cc:302] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:37.319203 4370 raft_consensus.cc:2781] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:37.319262 4370 raft_consensus.cc:482] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:37.319299 4370 raft_consensus.cc:3037] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.320292 4370 raft_consensus.cc:504] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.320574 4472 tablet_bootstrap.cc:492] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:37.320652 4472 ts_tablet_manager.cc:1367] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:37.320654 4370 leader_election.cc:288] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:37.321077 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "907ccfaebc3c4181bda0d3f03aba050c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:37.321125 4472 raft_consensus.cc:348] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.321188 3019 raft_consensus.cc:3037] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.321218 4472 raft_consensus.cc:374] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.321179 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "907ccfaebc3c4181bda0d3f03aba050c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:37.321254 4472 raft_consensus.cc:729] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.321313 3150 raft_consensus.cc:3037] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.321375 4472 consensus_queue.cc:260] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.321617 4472 ts_tablet_manager.cc:1398] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.322041 3019 raft_consensus.cc:2454] T 907ccfaebc3c4181bda0d3f03aba050c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:37.322319 3150 raft_consensus.cc:2454] T 907ccfaebc3c4181bda0d3f03aba050c P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:37.322352 3216 leader_election.cc:302] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:37.322511 4370 raft_consensus.cc:2781] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:37.322589 4370 raft_consensus.cc:686] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:37.322716 4370 consensus_queue.cc:237] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.324689 32726 catalog_manager.cc:5484] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:37.358510 4194 raft_consensus.cc:482] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:37.358633 4194 raft_consensus.cc:504] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.359050 4194 leader_election.cc:288] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:37.359351 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529244fd7ea14a60978d3f8fa6ad6efd" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:37.359418 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529244fd7ea14a60978d3f8fa6ad6efd" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:37.359490 3019 raft_consensus.cc:2454] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:37.359555 3283 raft_consensus.cc:2454] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:37.359771 3084 leader_election.cc:302] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:37.359969 4194 raft_consensus.cc:2781] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:37.360035 4194 raft_consensus.cc:482] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:37.360072 4194 raft_consensus.cc:3037] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.361156 4194 raft_consensus.cc:504] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.361445 4194 leader_election.cc:288] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:37.361732 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529244fd7ea14a60978d3f8fa6ad6efd" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:37.361752 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "529244fd7ea14a60978d3f8fa6ad6efd" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:37.361847 3019 raft_consensus.cc:3037] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.361860 3283 raft_consensus.cc:3037] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.362809 3283 raft_consensus.cc:2454] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:37.362866 3019 raft_consensus.cc:2454] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:37.363093 3085 leader_election.cc:302] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:37.363313 4194 raft_consensus.cc:2781] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:37.363435 4194 raft_consensus.cc:686] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:37.363582 4194 consensus_queue.cc:237] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.365675 32731 catalog_manager.cc:5484] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:37.376215 4194 raft_consensus.cc:482] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:37.376338 4194 raft_consensus.cc:504] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.376695 4194 leader_election.cc:288] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:37.377002 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f808844727624864a0da274de95c2b05" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:37.377041 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f808844727624864a0da274de95c2b05" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:37.377133 3019 raft_consensus.cc:2454] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:37.377161 2884 raft_consensus.cc:2454] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:37.377399 3084 leader_election.cc:302] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:37.377630 4194 raft_consensus.cc:2781] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:37.377686 4194 raft_consensus.cc:482] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:37.377734 4194 raft_consensus.cc:3037] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.378535 4194 raft_consensus.cc:504] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.378899 4194 leader_election.cc:288] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:37.379163 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f808844727624864a0da274de95c2b05" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:37.379292 3019 raft_consensus.cc:3037] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.379268 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f808844727624864a0da274de95c2b05" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:37.379385 2884 raft_consensus.cc:3037] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.380088 3019 raft_consensus.cc:2454] T f808844727624864a0da274de95c2b05 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:37.380357 2884 raft_consensus.cc:2454] T f808844727624864a0da274de95c2b05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:37.380558 3084 leader_election.cc:302] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:37.380731 4194 raft_consensus.cc:2781] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:37.380821 4194 raft_consensus.cc:686] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:37.380970 4194 consensus_queue.cc:237] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.385500 32726 catalog_manager.cc:5484] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:37.402082 4468 raft_consensus.cc:482] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:37.402191 4468 raft_consensus.cc:504] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.402638 4468 leader_election.cc:288] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:37.402946 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b985ba4b9f534827919cbb24daa4de0e" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:37.403105 409 raft_consensus.cc:2379] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 452ecbc085e54db49c3b01c20a96f860 in current term 1: Already voted for candidate da3f8160557e454f8547f45e80c2f013 in this term.
I20240502 14:00:37.403250 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b985ba4b9f534827919cbb24daa4de0e" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:37.403628 2952 leader_election.cc:302] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860; no voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e
I20240502 14:00:37.403802 4468 raft_consensus.cc:3037] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:37.404771 4468 raft_consensus.cc:2726] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 14:00:37.475543 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:37.480405 3283 raft_consensus.cc:1261] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:37.480710 3019 raft_consensus.cc:1261] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:37.481209 4466 consensus_queue.cc:1035] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.481431 4194 consensus_queue.cc:1035] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.484022 32726 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet 529244fd7ea14a60978d3f8fa6ad6efd (table rebalancer_dynamic_table [id=c3e4d6b3e7414735b6367d0e3bd2d3e6]). Expected version 1 got 0
I20240502 14:00:37.484510 4417 tablet.cc:1722] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:37.484823 4417 tablet.cc:1620] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:37.485836 32731 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 529244fd7ea14a60978d3f8fa6ad6efd (table rebalancer_dynamic_table [id=c3e4d6b3e7414735b6367d0e3bd2d3e6]). Expected version 1 got 0
I20240502 14:00:37.486055 4419 tablet.cc:1722] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:37.486321 4419 tablet.cc:1620] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:37.490749 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=c3e4d6b3e7414735b6367d0e3bd2d3e6] alter complete (version 1)
I20240502 14:00:37.491667 32732 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 529244fd7ea14a60978d3f8fa6ad6efd (table rebalancer_dynamic_table [id=c3e4d6b3e7414735b6367d0e3bd2d3e6]). Expected version 1 got 0
I20240502 14:00:37.492537 4458 tablet.cc:1722] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:37.492817 4458 tablet.cc:1620] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:37.597324 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:37.599355 4370 consensus_queue.cc:1035] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.603717 4370 consensus_queue.cc:1035] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.606557 4417 tablet.cc:1722] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:37.606896 4417 tablet.cc:1620] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:37.607208 4458 tablet.cc:1722] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:37.607618 4458 tablet.cc:1620] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:37.607735 4419 tablet.cc:1722] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:37.608067 4419 tablet.cc:1620] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:37.613341 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=c3e4d6b3e7414735b6367d0e3bd2d3e6] alter complete (version 2)
I20240502 14:00:37.717466 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:37.722939 4417 tablet.cc:1722] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:37.723718 4458 tablet.cc:1722] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:37.723781 4419 tablet.cc:1722] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:37.728188 4317 consensus_queue.cc:1035] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.731174 4370 consensus_queue.cc:1035] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.729828 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=c3e4d6b3e7414735b6367d0e3bd2d3e6] alter complete (version 3)
I20240502 14:00:37.832846 32732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:37.833012 32732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:37.835263 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 529244fd7ea14a60978d3f8fa6ad6efd
I20240502 14:00:37.835997 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 529244fd7ea14a60978d3f8fa6ad6efd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:37 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:37.836066 2997 tablet_service.cc:1514] Processing DeleteTablet for tablet 529244fd7ea14a60978d3f8fa6ad6efd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:37 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:37.836336 4508 tablet_replica.cc:330] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:37.836349 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet 529244fd7ea14a60978d3f8fa6ad6efd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:37 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:37.836485 4508 raft_consensus.cc:2229] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:37.836666 4509 tablet_replica.cc:330] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:37.836659 4510 tablet_replica.cc:330] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:37.836836 4508 raft_consensus.cc:2258] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:37.836840 4509 raft_consensus.cc:2229] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:37.836803 4510 raft_consensus.cc:2229] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:37.836980 4509 raft_consensus.cc:2258] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:37.837044 4510 raft_consensus.cc:2258] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:37.837604 4508 ts_tablet_manager.cc:1875] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:37.837750 4510 ts_tablet_manager.cc:1875] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:37.837847 4509 ts_tablet_manager.cc:1875] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:37.840059 4508 ts_tablet_manager.cc:1888] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:37.840157 4508 log.cc:1196] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/529244fd7ea14a60978d3f8fa6ad6efd
I20240502 14:00:37.840097 4509 ts_tablet_manager.cc:1888] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:37.840250 4509 log.cc:1196] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/529244fd7ea14a60978d3f8fa6ad6efd
I20240502 14:00:37.840473 4508 ts_tablet_manager.cc:1909] T 529244fd7ea14a60978d3f8fa6ad6efd P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:37.840545 4509 ts_tablet_manager.cc:1909] T 529244fd7ea14a60978d3f8fa6ad6efd P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:37.841307 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 529244fd7ea14a60978d3f8fa6ad6efd (table rebalancer_dynamic_table_new_name [id=c3e4d6b3e7414735b6367d0e3bd2d3e6]) successfully deleted
I20240502 14:00:37.841351 4510 ts_tablet_manager.cc:1888] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:37.841421 4510 log.cc:1196] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/529244fd7ea14a60978d3f8fa6ad6efd
I20240502 14:00:37.841677 4510 ts_tablet_manager.cc:1909] T 529244fd7ea14a60978d3f8fa6ad6efd P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:37.842319 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 529244fd7ea14a60978d3f8fa6ad6efd (table rebalancer_dynamic_table_new_name [id=c3e4d6b3e7414735b6367d0e3bd2d3e6]) successfully deleted
I20240502 14:00:37.844013 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 529244fd7ea14a60978d3f8fa6ad6efd (table rebalancer_dynamic_table_new_name [id=c3e4d6b3e7414735b6367d0e3bd2d3e6]) successfully deleted
I20240502 14:00:37.921267 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:37.927574 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 094aa25b08b8498f9ae4493cafd289e3 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.927662 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 094aa25b08b8498f9ae4493cafd289e3 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.927737 784 tablet_service.cc:1467] Processing CreateTablet for tablet 094aa25b08b8498f9ae4493cafd289e3 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.927943 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 094aa25b08b8498f9ae4493cafd289e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.928002 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 094aa25b08b8498f9ae4493cafd289e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.928054 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 094aa25b08b8498f9ae4493cafd289e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.929927 4511 tablet_bootstrap.cc:492] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:37.930647 522 tablet_service.cc:1467] Processing CreateTablet for tablet b20d05c7433e4f56832233e2fc9adc84 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.930765 4512 tablet_bootstrap.cc:492] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:37.930929 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b20d05c7433e4f56832233e2fc9adc84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.930994 4511 tablet_bootstrap.cc:654] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.931684 4512 tablet_bootstrap.cc:654] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.932142 2864 tablet_service.cc:1467] Processing CreateTablet for tablet b20d05c7433e4f56832233e2fc9adc84 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.932364 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b20d05c7433e4f56832233e2fc9adc84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.932389 4513 tablet_bootstrap.cc:492] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:37.933522 4513 tablet_bootstrap.cc:654] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.934384 4512 tablet_bootstrap.cc:492] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:37.934346 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd00d821b5d4b35ada028a8c83d0ed7 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.934455 4512 ts_tablet_manager.cc:1367] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:37.934526 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd00d821b5d4b35ada028a8c83d0ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.934881 4512 raft_consensus.cc:348] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:37.934999 4512 raft_consensus.cc:374] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.935036 4512 raft_consensus.cc:729] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.935158 4512 consensus_queue.cc:260] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:37.935443 4512 ts_tablet_manager.cc:1398] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.935890 4513 tablet_bootstrap.cc:492] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:37.935983 4513 ts_tablet_manager.cc:1367] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:37.936412 4513 raft_consensus.cc:348] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:37.936507 4513 raft_consensus.cc:374] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.936542 4513 raft_consensus.cc:729] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.936671 4513 consensus_queue.cc:260] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:37.937052 4513 ts_tablet_manager.cc:1398] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.936185 4518 tablet_bootstrap.cc:492] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:37.938689 4518 tablet_bootstrap.cc:654] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.938943 3263 tablet_service.cc:1467] Processing CreateTablet for tablet b20d05c7433e4f56832233e2fc9adc84 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.939210 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b20d05c7433e4f56832233e2fc9adc84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.940567 4516 tablet_bootstrap.cc:492] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:37.941009 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd00d821b5d4b35ada028a8c83d0ed7 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.941299 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd00d821b5d4b35ada028a8c83d0ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.941884 4516 tablet_bootstrap.cc:654] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.943373 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd00d821b5d4b35ada028a8c83d0ed7 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:37.943652 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd00d821b5d4b35ada028a8c83d0ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.943899 4512 tablet_bootstrap.cc:492] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:37.944998 4512 tablet_bootstrap.cc:654] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.946190 4511 tablet_bootstrap.cc:492] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:37.946103 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:37.946293 4511 ts_tablet_manager.cc:1367] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 14:00:37.946672 4512 tablet_bootstrap.cc:492] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:37.946764 4512 ts_tablet_manager.cc:1367] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:37.946774 4511 raft_consensus.cc:348] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:37.946897 4511 raft_consensus.cc:374] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.946939 4511 raft_consensus.cc:729] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.947063 4511 consensus_queue.cc:260] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:37.947254 4512 raft_consensus.cc:348] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.947362 4512 raft_consensus.cc:374] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.947376 4511 ts_tablet_manager.cc:1398] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.947400 4512 raft_consensus.cc:729] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.947554 4511 tablet_bootstrap.cc:492] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:37.947520 4512 consensus_queue.cc:260] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.947837 4512 ts_tablet_manager.cc:1398] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.948508 4511 tablet_bootstrap.cc:654] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.949035 4518 tablet_bootstrap.cc:492] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:37.949147 4518 ts_tablet_manager.cc:1367] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 14:00:37.949664 4518 raft_consensus.cc:348] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:37.949782 4518 raft_consensus.cc:374] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.949819 4518 raft_consensus.cc:729] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.949934 4518 consensus_queue.cc:260] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:37.950292 4518 ts_tablet_manager.cc:1398] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.950443 4518 tablet_bootstrap.cc:492] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:37.951556 4518 tablet_bootstrap.cc:654] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.952353 4511 tablet_bootstrap.cc:492] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:37.952442 4511 ts_tablet_manager.cc:1367] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:37.952605 4516 tablet_bootstrap.cc:492] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:37.952699 4516 ts_tablet_manager.cc:1367] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 14:00:37.953148 4511 raft_consensus.cc:348] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:37.953182 4516 raft_consensus.cc:348] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:37.953531 4516 raft_consensus.cc:374] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.953575 4516 raft_consensus.cc:729] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.953701 4516 consensus_queue.cc:260] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:37.953323 4511 raft_consensus.cc:374] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.953802 4511 raft_consensus.cc:729] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.953842 3132 tablet_service.cc:1467] Processing CreateTablet for tablet ff568d27a2f14638abff8d89e95e9763 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c2e4dccff65d45f3b5f2e0a5e8309e93]), partition=
I20240502 14:00:37.953945 4511 consensus_queue.cc:260] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:37.953980 4516 ts_tablet_manager.cc:1398] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.954103 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff568d27a2f14638abff8d89e95e9763. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.954236 4511 ts_tablet_manager.cc:1398] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:37.954512 4511 tablet_bootstrap.cc:492] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:37.956609 2866 tablet_service.cc:1467] Processing CreateTablet for tablet ff568d27a2f14638abff8d89e95e9763 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c2e4dccff65d45f3b5f2e0a5e8309e93]), partition=
I20240502 14:00:37.956812 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff568d27a2f14638abff8d89e95e9763. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.957506 4194 consensus_queue.cc:1035] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.957584 3264 tablet_service.cc:1467] Processing CreateTablet for tablet ff568d27a2f14638abff8d89e95e9763 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c2e4dccff65d45f3b5f2e0a5e8309e93]), partition=
I20240502 14:00:37.957984 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff568d27a2f14638abff8d89e95e9763. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:37.960511 4524 tablet_bootstrap.cc:492] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:37.961138 4518 tablet_bootstrap.cc:492] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:37.961220 4518 ts_tablet_manager.cc:1367] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:00:37.961624 4518 raft_consensus.cc:348] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.961761 4518 raft_consensus.cc:374] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.961796 4518 raft_consensus.cc:729] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.961911 4518 consensus_queue.cc:260] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.962069 4511 tablet_bootstrap.cc:654] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.962209 4518 ts_tablet_manager.cc:1398] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.962364 4518 tablet_bootstrap.cc:492] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:37.963259 4518 tablet_bootstrap.cc:654] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.964571 4467 consensus_queue.cc:1035] T f808844727624864a0da274de95c2b05 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:37.965334 4524 tablet_bootstrap.cc:654] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.966791 4518 tablet_bootstrap.cc:492] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:37.966843 4511 tablet_bootstrap.cc:492] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:37.966882 4518 ts_tablet_manager.cc:1367] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:37.966924 4511 ts_tablet_manager.cc:1367] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 14:00:37.967358 4518 raft_consensus.cc:348] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.967459 4518 raft_consensus.cc:374] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.967396 4511 raft_consensus.cc:348] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.967496 4518 raft_consensus.cc:729] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.967507 4511 raft_consensus.cc:374] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.967545 4511 raft_consensus.cc:729] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.967595 4518 consensus_queue.cc:260] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.967667 4511 consensus_queue.cc:260] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:37.967972 4511 ts_tablet_manager.cc:1398] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.968147 4511 tablet_bootstrap.cc:492] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:37.968183 4518 ts_tablet_manager.cc:1398] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.969046 4511 tablet_bootstrap.cc:654] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:37.970373 4511 tablet_bootstrap.cc:492] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:37.970464 4511 ts_tablet_manager.cc:1367] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:37.970762 4524 tablet_bootstrap.cc:492] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:37.970842 4524 ts_tablet_manager.cc:1367] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:37.970880 4511 raft_consensus.cc:348] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.970975 4511 raft_consensus.cc:374] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.971014 4511 raft_consensus.cc:729] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.971123 4511 consensus_queue.cc:260] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.971311 4524 raft_consensus.cc:348] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.971412 4524 raft_consensus.cc:374] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:37.971418 4511 ts_tablet_manager.cc:1398] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.971443 4524 raft_consensus.cc:729] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:37.971555 4524 consensus_queue.cc:260] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:37.971947 4524 ts_tablet_manager.cc:1398] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:38.030707 4467 raft_consensus.cc:482] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:38.030854 4467 raft_consensus.cc:504] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.031275 4467 leader_election.cc:288] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:38.031771 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff568d27a2f14638abff8d89e95e9763" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:38.031929 3283 raft_consensus.cc:2454] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:38.032825 3085 leader_election.cc:302] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:38.033522 4467 raft_consensus.cc:2781] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:38.033582 4467 raft_consensus.cc:482] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:38.033617 4467 raft_consensus.cc:3037] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.034906 4467 raft_consensus.cc:504] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.035584 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff568d27a2f14638abff8d89e95e9763" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:38.035708 3283 raft_consensus.cc:3037] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.035725 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff568d27a2f14638abff8d89e95e9763" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:38.035887 2886 raft_consensus.cc:2454] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:38.036157 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff568d27a2f14638abff8d89e95e9763" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:38.036233 2886 raft_consensus.cc:3037] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.036737 3283 raft_consensus.cc:2454] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:38.037138 2886 raft_consensus.cc:2454] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:38.039140 3085 leader_election.cc:302] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:38.039307 4194 raft_consensus.cc:2781] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:38.039363 4194 raft_consensus.cc:686] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:38.039486 4194 consensus_queue.cc:237] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.041314 4467 leader_election.cc:288] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:38.042243 32733 catalog_manager.cc:5484] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.060287 4317 raft_consensus.cc:482] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:38.060443 4317 raft_consensus.cc:504] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:38.060963 4317 leader_election.cc:288] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:38.061285 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd00d821b5d4b35ada028a8c83d0ed7" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:38.061424 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd00d821b5d4b35ada028a8c83d0ed7" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:38.061455 3018 raft_consensus.cc:2454] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:38.061573 2886 raft_consensus.cc:2454] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:38.061805 3216 leader_election.cc:302] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:38.061990 4317 raft_consensus.cc:2781] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:38.062047 4317 raft_consensus.cc:482] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:38.062096 4317 raft_consensus.cc:3037] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.062919 4317 raft_consensus.cc:504] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:38.063271 4317 leader_election.cc:288] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:38.063597 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd00d821b5d4b35ada028a8c83d0ed7" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:38.063675 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd00d821b5d4b35ada028a8c83d0ed7" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:38.063719 3018 raft_consensus.cc:3037] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.063793 2886 raft_consensus.cc:3037] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.064635 3018 raft_consensus.cc:2454] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:38.064700 2886 raft_consensus.cc:2454] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:38.065022 3216 leader_election.cc:302] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:38.065222 4317 raft_consensus.cc:2781] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:38.065307 4317 raft_consensus.cc:686] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:38.065441 4317 consensus_queue.cc:237] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:38.067449 32733 catalog_manager.cc:5484] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.080030 4370 raft_consensus.cc:482] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:38.080147 4370 raft_consensus.cc:504] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:38.080526 4370 leader_election.cc:288] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:38.080909 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20d05c7433e4f56832233e2fc9adc84" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:38.081066 2886 raft_consensus.cc:2454] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:38.081101 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20d05c7433e4f56832233e2fc9adc84" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:38.081290 541 raft_consensus.cc:2454] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:38.081532 3219 leader_election.cc:302] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:38.081748 4370 raft_consensus.cc:2781] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:38.081794 4370 raft_consensus.cc:482] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:38.081818 4370 raft_consensus.cc:3037] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.082612 4370 raft_consensus.cc:504] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:38.082902 4370 leader_election.cc:288] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:38.083271 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20d05c7433e4f56832233e2fc9adc84" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:38.083382 2886 raft_consensus.cc:3037] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.084200 2886 raft_consensus.cc:2454] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:38.084446 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20d05c7433e4f56832233e2fc9adc84" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:38.084532 3219 leader_election.cc:302] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:38.084558 541 raft_consensus.cc:3037] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.084708 4370 raft_consensus.cc:2781] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:38.084779 4370 raft_consensus.cc:686] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:38.084916 4370 consensus_queue.cc:237] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:38.085510 541 raft_consensus.cc:2454] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:38.086931 32733 catalog_manager.cc:5484] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.143800 4533 raft_consensus.cc:482] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:38.144007 4533 raft_consensus.cc:504] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:38.144418 4533 leader_election.cc:288] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:38.144766 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:38.144830 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:38.144907 3018 raft_consensus.cc:2454] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:38.144982 805 raft_consensus.cc:2454] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:38.145217 3216 leader_election.cc:302] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:38.145416 4533 raft_consensus.cc:2781] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:38.145483 4533 raft_consensus.cc:482] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:38.145537 4533 raft_consensus.cc:3037] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.146486 4533 raft_consensus.cc:504] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:38.146836 4533 leader_election.cc:288] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:38.147171 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:38.147313 805 raft_consensus.cc:3037] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.147336 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:38.147459 3018 raft_consensus.cc:3037] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.148211 805 raft_consensus.cc:2454] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:38.148382 3018 raft_consensus.cc:2454] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:38.148509 3219 leader_election.cc:302] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:38.148715 4533 raft_consensus.cc:2781] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:38.148833 4533 raft_consensus.cc:686] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:38.149016 4533 consensus_queue.cc:237] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:38.151134 32733 catalog_manager.cc:5484] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.161985 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:38.165766 3283 raft_consensus.cc:1261] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:38.166021 2886 raft_consensus.cc:1261] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:38.167124 4194 consensus_queue.cc:1035] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:38.167419 4194 consensus_queue.cc:1035] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:38.169741 4417 tablet.cc:1722] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:38.170063 4417 tablet.cc:1620] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:38.170183 32731 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet ff568d27a2f14638abff8d89e95e9763 (table rebalancer_dynamic_table [id=c2e4dccff65d45f3b5f2e0a5e8309e93]). Expected version 1 got 0
I20240502 14:00:38.170893 32732 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet ff568d27a2f14638abff8d89e95e9763 (table rebalancer_dynamic_table [id=c2e4dccff65d45f3b5f2e0a5e8309e93]). Expected version 1 got 0
I20240502 14:00:38.171914 4458 tablet.cc:1722] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:38.172233 4458 tablet.cc:1620] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:38.173126 4539 tablet.cc:1722] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:38.173486 4539 tablet.cc:1620] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:38.174543 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=c2e4dccff65d45f3b5f2e0a5e8309e93] alter complete (version 1)
I20240502 14:00:38.174712 32733 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet ff568d27a2f14638abff8d89e95e9763 (table rebalancer_dynamic_table [id=c2e4dccff65d45f3b5f2e0a5e8309e93]). Expected version 1 got 0
I20240502 14:00:38.278771 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:38.284008 4417 tablet.cc:1722] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:38.284372 4417 tablet.cc:1620] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:38.284577 4539 tablet.cc:1722] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:38.284909 4539 tablet.cc:1620] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:38.285408 4458 tablet.cc:1722] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:38.285706 4458 tablet.cc:1620] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:38.290695 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=c2e4dccff65d45f3b5f2e0a5e8309e93] alter complete (version 2)
I20240502 14:00:38.392941 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:38.401701 4417 tablet.cc:1722] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:38.402698 4458 tablet.cc:1722] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:38.403560 4539 tablet.cc:1722] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:38.407229 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=c2e4dccff65d45f3b5f2e0a5e8309e93] alter complete (version 3)
I20240502 14:00:38.508654 4370 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:38.509605 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:38.509773 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:38.512535 4317 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:38.513015 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet ff568d27a2f14638abff8d89e95e9763
I20240502 14:00:38.513892 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet ff568d27a2f14638abff8d89e95e9763 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:38 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:38.514356 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet ff568d27a2f14638abff8d89e95e9763 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:38 UTC) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:38.515492 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet ff568d27a2f14638abff8d89e95e9763 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:38 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:38.515812 4548 tablet_replica.cc:330] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:38.515901 4547 tablet_replica.cc:330] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:38.516069 4548 raft_consensus.cc:2229] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.516119 4547 raft_consensus.cc:2229] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.516270 4548 raft_consensus.cc:2258] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.516341 4546 tablet_replica.cc:330] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:38.516392 4547 raft_consensus.cc:2258] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.516745 4546 raft_consensus.cc:2229] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:38.517030 4548 ts_tablet_manager.cc:1875] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:38.517031 4546 raft_consensus.cc:2258] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.517786 4546 ts_tablet_manager.cc:1875] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:38.517827 4547 ts_tablet_manager.cc:1875] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:38.519155 4548 ts_tablet_manager.cc:1888] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:38.519228 4548 log.cc:1196] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/ff568d27a2f14638abff8d89e95e9763
I20240502 14:00:38.519482 4548 ts_tablet_manager.cc:1909] T ff568d27a2f14638abff8d89e95e9763 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:38.520092 4546 ts_tablet_manager.cc:1888] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:38.520089 4547 ts_tablet_manager.cc:1888] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:38.520169 4547 log.cc:1196] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/ff568d27a2f14638abff8d89e95e9763
I20240502 14:00:38.520318 4546 log.cc:1196] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/ff568d27a2f14638abff8d89e95e9763
I20240502 14:00:38.520303 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet ff568d27a2f14638abff8d89e95e9763 (table rebalancer_dynamic_table_new_name [id=c2e4dccff65d45f3b5f2e0a5e8309e93]) successfully deleted
I20240502 14:00:38.520450 4547 ts_tablet_manager.cc:1909] T ff568d27a2f14638abff8d89e95e9763 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting consensus metadata
I20240502 14:00:38.520668 4546 ts_tablet_manager.cc:1909] T ff568d27a2f14638abff8d89e95e9763 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:38.521126 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet ff568d27a2f14638abff8d89e95e9763 (table rebalancer_dynamic_table_new_name [id=c2e4dccff65d45f3b5f2e0a5e8309e93]) successfully deleted
I20240502 14:00:38.521319 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet ff568d27a2f14638abff8d89e95e9763 (table rebalancer_dynamic_table_new_name [id=c2e4dccff65d45f3b5f2e0a5e8309e93]) successfully deleted
I20240502 14:00:38.595374 4317 consensus_queue.cc:1035] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:38.601049 4370 consensus_queue.cc:1035] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:38.608680 4370 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:38.612450 4553 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:38.615250 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:38.620770 2997 tablet_service.cc:1467] Processing CreateTablet for tablet a8594195210c4391bbd56a4d696f8e00 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6ae252b5a8b240158c68f140f828a16f]), partition=
I20240502 14:00:38.621048 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8594195210c4391bbd56a4d696f8e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.621141 3132 tablet_service.cc:1467] Processing CreateTablet for tablet a8594195210c4391bbd56a4d696f8e00 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6ae252b5a8b240158c68f140f828a16f]), partition=
I20240502 14:00:38.621141 784 tablet_service.cc:1467] Processing CreateTablet for tablet a8594195210c4391bbd56a4d696f8e00 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6ae252b5a8b240158c68f140f828a16f]), partition=
I20240502 14:00:38.621377 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8594195210c4391bbd56a4d696f8e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.621376 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8594195210c4391bbd56a4d696f8e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.622787 4556 tablet_bootstrap.cc:492] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:38.624065 4556 tablet_bootstrap.cc:654] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.624504 4557 tablet_bootstrap.cc:492] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:38.625284 4557 tablet_bootstrap.cc:654] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.624289 4558 tablet_bootstrap.cc:492] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:38.627236 4558 tablet_bootstrap.cc:654] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.628110 4557 tablet_bootstrap.cc:492] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:38.628206 4557 ts_tablet_manager.cc:1367] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:38.628640 4557 raft_consensus.cc:348] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.628738 4557 raft_consensus.cc:374] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.628759 4557 raft_consensus.cc:729] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.628871 4557 consensus_queue.cc:260] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.629176 4557 ts_tablet_manager.cc:1398] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:38.629666 4558 tablet_bootstrap.cc:492] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:38.629761 4558 ts_tablet_manager.cc:1367] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:00:38.630808 4558 raft_consensus.cc:348] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.630944 4558 raft_consensus.cc:374] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.630973 4558 raft_consensus.cc:729] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.631323 4558 consensus_queue.cc:260] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.631703 4558 ts_tablet_manager.cc:1398] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:38.633365 4556 tablet_bootstrap.cc:492] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:38.633486 4556 ts_tablet_manager.cc:1367] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20240502 14:00:38.634594 4556 raft_consensus.cc:348] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.634721 4556 raft_consensus.cc:374] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.634752 4556 raft_consensus.cc:729] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.634943 4556 consensus_queue.cc:260] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.635301 4556 ts_tablet_manager.cc:1398] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:38.689097 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:38.695151 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 2df438b3678241d9a6aa6f0deb2cee2d (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=a6ece96546b04c28ab94cb79046f2008]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:38.695238 522 tablet_service.cc:1467] Processing CreateTablet for tablet 2df438b3678241d9a6aa6f0deb2cee2d (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=a6ece96546b04c28ab94cb79046f2008]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:38.695307 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 2df438b3678241d9a6aa6f0deb2cee2d (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=a6ece96546b04c28ab94cb79046f2008]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:38.695510 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2df438b3678241d9a6aa6f0deb2cee2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.695556 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2df438b3678241d9a6aa6f0deb2cee2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.695551 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2df438b3678241d9a6aa6f0deb2cee2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.697588 4562 tablet_bootstrap.cc:492] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:38.698015 386 tablet_service.cc:1467] Processing CreateTablet for tablet 6e088025427a40958d331d432906fd44 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=a6ece96546b04c28ab94cb79046f2008]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:38.698041 2997 tablet_service.cc:1467] Processing CreateTablet for tablet ed1ae34edc6b4b6c864a6b7659810475 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=a6ece96546b04c28ab94cb79046f2008]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:38.698215 4564 tablet_bootstrap.cc:492] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:38.698293 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed1ae34edc6b4b6c864a6b7659810475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.698293 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e088025427a40958d331d432906fd44. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.698601 4562 tablet_bootstrap.cc:654] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.699257 4564 tablet_bootstrap.cc:654] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.700183 4564 tablet_bootstrap.cc:492] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:38.700245 4564 ts_tablet_manager.cc:1367] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:38.700771 4556 tablet_bootstrap.cc:492] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:38.701052 4567 tablet_bootstrap.cc:492] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:38.701365 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 6e088025427a40958d331d432906fd44 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=a6ece96546b04c28ab94cb79046f2008]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:38.701623 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e088025427a40958d331d432906fd44. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.701901 4556 tablet_bootstrap.cc:654] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.702538 3130 tablet_service.cc:1467] Processing CreateTablet for tablet ed1ae34edc6b4b6c864a6b7659810475 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=a6ece96546b04c28ab94cb79046f2008]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:38.702646 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 6e088025427a40958d331d432906fd44 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=a6ece96546b04c28ab94cb79046f2008]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:38.702827 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed1ae34edc6b4b6c864a6b7659810475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.703032 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e088025427a40958d331d432906fd44. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.703760 3264 tablet_service.cc:1467] Processing CreateTablet for tablet ed1ae34edc6b4b6c864a6b7659810475 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=a6ece96546b04c28ab94cb79046f2008]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:38.704025 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed1ae34edc6b4b6c864a6b7659810475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.704763 4567 tablet_bootstrap.cc:654] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.705090 4558 tablet_bootstrap.cc:492] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:38.706043 4558 tablet_bootstrap.cc:654] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.706300 4564 raft_consensus.cc:348] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.706437 4564 raft_consensus.cc:374] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.706475 4564 raft_consensus.cc:729] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.706652 4564 consensus_queue.cc:260] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.706959 4564 ts_tablet_manager.cc:1398] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:00:38.707340 4564 tablet_bootstrap.cc:492] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:38.707794 4563 tablet_bootstrap.cc:492] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:38.708472 4564 tablet_bootstrap.cc:654] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.708504 4558 tablet_bootstrap.cc:492] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:38.708590 4558 ts_tablet_manager.cc:1367] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:38.708910 4563 tablet_bootstrap.cc:654] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.709064 4558 raft_consensus.cc:348] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.709177 4558 raft_consensus.cc:374] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.709213 4558 raft_consensus.cc:729] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.709338 4558 consensus_queue.cc:260] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.709602 4558 ts_tablet_manager.cc:1398] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:38.709770 4564 tablet_bootstrap.cc:492] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:38.709846 4564 ts_tablet_manager.cc:1367] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:38.710258 4564 raft_consensus.cc:348] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.710359 4564 raft_consensus.cc:374] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.710393 4564 raft_consensus.cc:729] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.710497 4564 consensus_queue.cc:260] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.710754 4564 ts_tablet_manager.cc:1398] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:38.710767 4567 tablet_bootstrap.cc:492] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:38.710856 4567 ts_tablet_manager.cc:1367] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:38.711305 4567 raft_consensus.cc:348] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:38.711467 4558 tablet_bootstrap.cc:492] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:38.711514 4567 raft_consensus.cc:374] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.711563 4567 raft_consensus.cc:729] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.711704 4567 consensus_queue.cc:260] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:38.712229 4562 tablet_bootstrap.cc:492] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:38.712323 4562 ts_tablet_manager.cc:1367] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:00:38.712393 4558 tablet_bootstrap.cc:654] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.712806 4562 raft_consensus.cc:348] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.712862 4567 ts_tablet_manager.cc:1398] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:38.712917 4562 raft_consensus.cc:374] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.712960 4562 raft_consensus.cc:729] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.713088 4562 consensus_queue.cc:260] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.713392 4562 ts_tablet_manager.cc:1398] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:38.713402 4563 tablet_bootstrap.cc:492] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:38.713510 4563 ts_tablet_manager.cc:1367] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 14:00:38.713989 4563 raft_consensus.cc:348] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.714088 4563 raft_consensus.cc:374] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.714110 4563 raft_consensus.cc:729] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.714242 4563 consensus_queue.cc:260] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.714502 4563 ts_tablet_manager.cc:1398] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:38.715773 4563 tablet_bootstrap.cc:492] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:38.716751 4563 tablet_bootstrap.cc:654] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:38.718360 4563 tablet_bootstrap.cc:492] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:38.718431 4563 ts_tablet_manager.cc:1367] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:38.718794 4563 raft_consensus.cc:348] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:38.718881 4563 raft_consensus.cc:374] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.718912 4563 raft_consensus.cc:729] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.719029 4563 consensus_queue.cc:260] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:38.719295 4563 ts_tablet_manager.cc:1398] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:38.719861 4556 tablet_bootstrap.cc:492] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:38.719971 4556 ts_tablet_manager.cc:1367] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.002s
I20240502 14:00:38.720510 4556 raft_consensus.cc:348] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.720631 4556 raft_consensus.cc:374] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.720665 4556 raft_consensus.cc:729] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.720834 4556 consensus_queue.cc:260] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.721783 4556 ts_tablet_manager.cc:1398] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:38.722606 4558 tablet_bootstrap.cc:492] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:38.722712 4558 ts_tablet_manager.cc:1367] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20240502 14:00:38.723209 4558 raft_consensus.cc:348] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:38.723310 4558 raft_consensus.cc:374] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:38.723347 4558 raft_consensus.cc:729] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:38.723488 4558 consensus_queue.cc:260] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:38.723798 4558 ts_tablet_manager.cc:1398] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:38.780583 4553 raft_consensus.cc:482] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:38.780709 4553 raft_consensus.cc:504] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.781227 4553 leader_election.cc:288] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:38.781513 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1ae34edc6b4b6c864a6b7659810475" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:38.781654 3018 raft_consensus.cc:2454] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:38.781948 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1ae34edc6b4b6c864a6b7659810475" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:38.782109 3150 raft_consensus.cc:2454] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:38.782115 3216 leader_election.cc:302] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:38.782303 4553 raft_consensus.cc:2781] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:38.782361 4553 raft_consensus.cc:482] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:38.782399 4553 raft_consensus.cc:3037] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.783469 4553 raft_consensus.cc:504] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.783861 4553 leader_election.cc:288] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:38.784206 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1ae34edc6b4b6c864a6b7659810475" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:38.784216 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed1ae34edc6b4b6c864a6b7659810475" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:38.784343 3018 raft_consensus.cc:3037] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.784341 3150 raft_consensus.cc:3037] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.785156 3150 raft_consensus.cc:2454] T ed1ae34edc6b4b6c864a6b7659810475 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:38.785241 3018 raft_consensus.cc:2454] T ed1ae34edc6b4b6c864a6b7659810475 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:38.785418 3219 leader_election.cc:302] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:38.785596 4553 raft_consensus.cc:2781] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:38.785710 4553 raft_consensus.cc:686] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:38.785857 4553 consensus_queue.cc:237] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.787307 4285 raft_consensus.cc:482] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:38.787410 4285 raft_consensus.cc:504] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.787850 4285 leader_election.cc:288] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:38.788105 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8594195210c4391bbd56a4d696f8e00" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:38.788199 3018 raft_consensus.cc:2454] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:38.788441 738 leader_election.cc:302] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:38.788604 4285 raft_consensus.cc:2781] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:38.788614 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8594195210c4391bbd56a4d696f8e00" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:38.788668 4285 raft_consensus.cc:482] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:38.788717 4285 raft_consensus.cc:3037] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.788727 3150 raft_consensus.cc:2454] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:38.789661 4285 raft_consensus.cc:504] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.789973 4285 leader_election.cc:288] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:38.790251 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8594195210c4391bbd56a4d696f8e00" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:38.790177 32733 catalog_manager.cc:5484] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.790360 3018 raft_consensus.cc:3037] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.791160 3018 raft_consensus.cc:2454] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:38.791440 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8594195210c4391bbd56a4d696f8e00" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:38.791447 738 leader_election.cc:302] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:38.791539 3150 raft_consensus.cc:3037] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.791644 4285 raft_consensus.cc:2781] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:38.791719 4285 raft_consensus.cc:686] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:38.791858 4285 consensus_queue.cc:237] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:38.792479 3150 raft_consensus.cc:2454] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:38.794011 32733 catalog_manager.cc:5484] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.874922 4381 raft_consensus.cc:482] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:38.875056 4381 raft_consensus.cc:504] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.875475 4381 leader_election.cc:288] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:38.875838 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df438b3678241d9a6aa6f0deb2cee2d" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:38.875949 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df438b3678241d9a6aa6f0deb2cee2d" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:38.875994 2886 raft_consensus.cc:2454] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:38.876088 3284 raft_consensus.cc:2454] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:38.876322 477 leader_election.cc:302] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:38.876593 4381 raft_consensus.cc:2781] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:38.876667 4381 raft_consensus.cc:482] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:38.876699 4381 raft_consensus.cc:3037] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.877746 4381 raft_consensus.cc:504] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.878098 4381 leader_election.cc:288] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:38.878458 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df438b3678241d9a6aa6f0deb2cee2d" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:38.878499 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df438b3678241d9a6aa6f0deb2cee2d" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:38.878563 3284 raft_consensus.cc:3037] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.878608 2886 raft_consensus.cc:3037] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.879359 3284 raft_consensus.cc:2454] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:38.879467 2886 raft_consensus.cc:2454] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:38.879622 475 leader_election.cc:302] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:38.879781 4381 raft_consensus.cc:2781] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:38.879887 4381 raft_consensus.cc:686] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:38.880026 4381 consensus_queue.cc:237] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:38.882141 32733 catalog_manager.cc:5484] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.883759 4466 raft_consensus.cc:482] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:38.883878 4466 raft_consensus.cc:504] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:38.884379 4466 leader_election.cc:288] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:38.884732 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e088025427a40958d331d432906fd44" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:38.884792 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e088025427a40958d331d432906fd44" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:38.884852 409 raft_consensus.cc:2454] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:38.884915 2886 raft_consensus.cc:2454] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:38.885102 3086 leader_election.cc:302] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:38.885298 4466 raft_consensus.cc:2781] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:38.885368 4466 raft_consensus.cc:482] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:38.885407 4466 raft_consensus.cc:3037] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.886376 4466 raft_consensus.cc:504] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:38.886651 4466 leader_election.cc:288] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:38.886941 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e088025427a40958d331d432906fd44" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:38.886947 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e088025427a40958d331d432906fd44" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:38.887037 2886 raft_consensus.cc:3037] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.887053 409 raft_consensus.cc:3037] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:38.887779 409 raft_consensus.cc:2454] T 6e088025427a40958d331d432906fd44 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:38.887787 2886 raft_consensus.cc:2454] T 6e088025427a40958d331d432906fd44 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:38.888118 3086 leader_election.cc:302] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:38.888342 4466 raft_consensus.cc:2781] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:38.888433 4466 raft_consensus.cc:686] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:38.888571 4466 consensus_queue.cc:237] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:38.890480 32733 catalog_manager.cc:5484] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.917204 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:38.921059 3150 raft_consensus.cc:1261] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:38.921468 4540 consensus_queue.cc:1035] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:38.922195 3018 raft_consensus.cc:1261] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:38.922879 4540 consensus_queue.cc:1035] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:38.924289 32733 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet a8594195210c4391bbd56a4d696f8e00 (table rebalancer_dynamic_table [id=6ae252b5a8b240158c68f140f828a16f]). Expected version 1 got 0
I20240502 14:00:38.925619 4584 tablet.cc:1722] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:38.926294 4585 tablet.cc:1722] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:38.926750 4585 tablet.cc:1620] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:38.927074 32733 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet a8594195210c4391bbd56a4d696f8e00 (table rebalancer_dynamic_table [id=6ae252b5a8b240158c68f140f828a16f]). Expected version 1 got 0
I20240502 14:00:38.927486 32731 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet a8594195210c4391bbd56a4d696f8e00 (table rebalancer_dynamic_table [id=6ae252b5a8b240158c68f140f828a16f]). Expected version 1 got 0
I20240502 14:00:38.928491 32731 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet a8594195210c4391bbd56a4d696f8e00 (table rebalancer_dynamic_table [id=6ae252b5a8b240158c68f140f828a16f]). Expected version 1 got 0
I20240502 14:00:38.928874 4586 tablet.cc:1722] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:38.929179 4586 tablet.cc:1620] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:38.929361 32731 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet a8594195210c4391bbd56a4d696f8e00 (table rebalancer_dynamic_table [id=6ae252b5a8b240158c68f140f828a16f]). Expected version 1 got 0
I20240502 14:00:38.931069 4584 tablet.cc:1620] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:38.933081 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=6ae252b5a8b240158c68f140f828a16f] alter complete (version 1)
I20240502 14:00:39.034677 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:39.041025 4584 tablet.cc:1722] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:39.041457 4584 tablet.cc:1620] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:39.041911 4586 tablet.cc:1722] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:39.042179 4586 tablet.cc:1620] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:39.043031 4585 tablet.cc:1722] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:39.043295 4585 tablet.cc:1620] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:39.047495 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=6ae252b5a8b240158c68f140f828a16f] alter complete (version 2)
I20240502 14:00:39.150382 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:39.157728 4585 tablet.cc:1722] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:39.157269 4584 tablet.cc:1722] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:39.162302 4586 tablet.cc:1722] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:39.162873 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=6ae252b5a8b240158c68f140f828a16f] alter complete (version 3)
I20240502 14:00:39.256911 4381 consensus_queue.cc:1035] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:39.261855 4541 consensus_queue.cc:1035] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:39.265986 32733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:39.266139 32733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:39.268819 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet a8594195210c4391bbd56a4d696f8e00
I20240502 14:00:39.269416 2997 tablet_service.cc:1514] Processing DeleteTablet for tablet a8594195210c4391bbd56a4d696f8e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:39 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:39.269490 784 tablet_service.cc:1514] Processing DeleteTablet for tablet a8594195210c4391bbd56a4d696f8e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:39 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:39.269564 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet a8594195210c4391bbd56a4d696f8e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:39 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:39.269814 4594 tablet_replica.cc:330] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:39.269882 4596 tablet_replica.cc:330] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:39.269999 4594 raft_consensus.cc:2229] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:39.270020 4595 tablet_replica.cc:330] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:39.270073 4596 raft_consensus.cc:2229] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:39.270126 4594 raft_consensus.cc:2258] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:39.270155 4595 raft_consensus.cc:2229] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:39.270424 4595 raft_consensus.cc:2258] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:39.270448 4596 raft_consensus.cc:2258] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:39.270828 4594 ts_tablet_manager.cc:1875] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:39.271059 4595 ts_tablet_manager.cc:1875] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:39.271111 4596 ts_tablet_manager.cc:1875] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:39.272851 4595 ts_tablet_manager.cc:1888] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:39.272922 4595 log.cc:1196] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/a8594195210c4391bbd56a4d696f8e00
I20240502 14:00:39.273211 4595 ts_tablet_manager.cc:1909] T a8594195210c4391bbd56a4d696f8e00 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:39.273361 4594 ts_tablet_manager.cc:1888] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:39.273361 4596 ts_tablet_manager.cc:1888] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:39.273442 4596 log.cc:1196] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/a8594195210c4391bbd56a4d696f8e00
I20240502 14:00:39.273442 4594 log.cc:1196] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/a8594195210c4391bbd56a4d696f8e00
I20240502 14:00:39.273758 4594 ts_tablet_manager.cc:1909] T a8594195210c4391bbd56a4d696f8e00 P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:39.273890 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet a8594195210c4391bbd56a4d696f8e00 (table rebalancer_dynamic_table_new_name [id=6ae252b5a8b240158c68f140f828a16f]) successfully deleted
I20240502 14:00:39.273782 4596 ts_tablet_manager.cc:1909] T a8594195210c4391bbd56a4d696f8e00 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:39.274483 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet a8594195210c4391bbd56a4d696f8e00 (table rebalancer_dynamic_table_new_name [id=6ae252b5a8b240158c68f140f828a16f]) successfully deleted
I20240502 14:00:39.274678 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet a8594195210c4391bbd56a4d696f8e00 (table rebalancer_dynamic_table_new_name [id=6ae252b5a8b240158c68f140f828a16f]) successfully deleted
I20240502 14:00:39.284093 4194 consensus_queue.cc:1035] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:39.287130 4598 consensus_queue.cc:1035] T 6e088025427a40958d331d432906fd44 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:39.303428 4370 consensus_queue.cc:1035] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:39.306818 4553 consensus_queue.cc:1035] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:39.370790 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:39.376300 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 8bf7896d6e664a698e9271340c24c908 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b262cd0ebbae4519adc79bbb9f1f269b]), partition=
I20240502 14:00:39.376621 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bf7896d6e664a698e9271340c24c908. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.378520 4603 tablet_bootstrap.cc:492] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:39.378674 784 tablet_service.cc:1467] Processing CreateTablet for tablet 8bf7896d6e664a698e9271340c24c908 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b262cd0ebbae4519adc79bbb9f1f269b]), partition=
I20240502 14:00:39.378926 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bf7896d6e664a698e9271340c24c908. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.379567 4603 tablet_bootstrap.cc:654] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.380293 653 tablet_service.cc:1467] Processing CreateTablet for tablet 8bf7896d6e664a698e9271340c24c908 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b262cd0ebbae4519adc79bbb9f1f269b]), partition=
I20240502 14:00:39.380546 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bf7896d6e664a698e9271340c24c908. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.381439 4605 tablet_bootstrap.cc:492] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:39.382606 4605 tablet_bootstrap.cc:654] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.382844 4606 tablet_bootstrap.cc:492] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:39.383625 4603 tablet_bootstrap.cc:492] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:39.383728 4603 ts_tablet_manager.cc:1367] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 14:00:39.384140 4606 tablet_bootstrap.cc:654] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.384169 4603 raft_consensus.cc:348] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:39.384330 4603 raft_consensus.cc:374] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.384395 4603 raft_consensus.cc:729] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.384543 4603 consensus_queue.cc:260] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:39.384840 4603 ts_tablet_manager.cc:1398] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:39.385704 4605 tablet_bootstrap.cc:492] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:39.385841 4605 ts_tablet_manager.cc:1367] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:39.386512 4605 raft_consensus.cc:348] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:39.386643 4605 raft_consensus.cc:374] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.386708 4605 raft_consensus.cc:729] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.386849 4605 consensus_queue.cc:260] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:39.387547 4605 ts_tablet_manager.cc:1398] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:39.392531 4606 tablet_bootstrap.cc:492] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:39.392637 4606 ts_tablet_manager.cc:1367] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:39.393041 4606 raft_consensus.cc:348] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:39.393142 4606 raft_consensus.cc:374] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.393177 4606 raft_consensus.cc:729] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.393352 4606 consensus_queue.cc:260] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:39.394709 4606 ts_tablet_manager.cc:1398] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:39.400035 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:39.406867 3132 tablet_service.cc:1467] Processing CreateTablet for tablet c80435eecda64ecea7864522f5a1dd4c (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=6716820a9c7447f292faab28186dd3ae]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:39.407198 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c80435eecda64ecea7864522f5a1dd4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.407203 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 2ffed38329ca4e339a849c5ab3b09e8b (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=6716820a9c7447f292faab28186dd3ae]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:39.407495 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ffed38329ca4e339a849c5ab3b09e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.408007 3264 tablet_service.cc:1467] Processing CreateTablet for tablet c80435eecda64ecea7864522f5a1dd4c (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=6716820a9c7447f292faab28186dd3ae]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:39.408171 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 2ffed38329ca4e339a849c5ab3b09e8b (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=6716820a9c7447f292faab28186dd3ae]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:39.408255 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c80435eecda64ecea7864522f5a1dd4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.408442 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ffed38329ca4e339a849c5ab3b09e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.409878 4610 tablet_bootstrap.cc:492] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:39.409842 522 tablet_service.cc:1467] Processing CreateTablet for tablet c80435eecda64ecea7864522f5a1dd4c (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=6716820a9c7447f292faab28186dd3ae]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:39.410097 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c80435eecda64ecea7864522f5a1dd4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.410588 3263 tablet_service.cc:1467] Processing CreateTablet for tablet d9c57d60e9354df0930fcc0b3df33f86 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=6716820a9c7447f292faab28186dd3ae]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:39.410822 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c57d60e9354df0930fcc0b3df33f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.410830 4610 tablet_bootstrap.cc:654] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.412696 653 tablet_service.cc:1467] Processing CreateTablet for tablet d9c57d60e9354df0930fcc0b3df33f86 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=6716820a9c7447f292faab28186dd3ae]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:39.412992 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c57d60e9354df0930fcc0b3df33f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.413064 4609 tablet_bootstrap.cc:492] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:39.413952 4613 tablet_bootstrap.cc:492] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:39.414149 4609 tablet_bootstrap.cc:654] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.415234 4606 tablet_bootstrap.cc:492] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:39.415405 4603 tablet_bootstrap.cc:492] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:39.416239 4606 tablet_bootstrap.cc:654] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.416363 4603 tablet_bootstrap.cc:654] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.416419 3130 tablet_service.cc:1467] Processing CreateTablet for tablet d9c57d60e9354df0930fcc0b3df33f86 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=6716820a9c7447f292faab28186dd3ae]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:39.416702 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c57d60e9354df0930fcc0b3df33f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.417155 4613 tablet_bootstrap.cc:654] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.417941 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 2ffed38329ca4e339a849c5ab3b09e8b (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=6716820a9c7447f292faab28186dd3ae]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:39.418165 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ffed38329ca4e339a849c5ab3b09e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.418305 4606 tablet_bootstrap.cc:492] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:39.418392 4606 ts_tablet_manager.cc:1367] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:39.418843 4606 raft_consensus.cc:348] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:39.418958 4606 raft_consensus.cc:374] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.418998 4606 raft_consensus.cc:729] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.419118 4606 consensus_queue.cc:260] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:39.419431 4606 ts_tablet_manager.cc:1398] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:39.422641 4609 tablet_bootstrap.cc:492] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:39.422785 4609 ts_tablet_manager.cc:1367] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:39.422956 4610 tablet_bootstrap.cc:492] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:39.423050 4610 ts_tablet_manager.cc:1367] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 14:00:39.423383 4609 raft_consensus.cc:348] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:39.423501 4609 raft_consensus.cc:374] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.423470 4610 raft_consensus.cc:348] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:39.423534 4609 raft_consensus.cc:729] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.423569 4610 raft_consensus.cc:374] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.423597 4610 raft_consensus.cc:729] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.423713 4610 consensus_queue.cc:260] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:39.424079 4610 ts_tablet_manager.cc:1398] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:39.423799 4609 consensus_queue.cc:260] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:39.424571 4609 ts_tablet_manager.cc:1398] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:39.424978 4609 tablet_bootstrap.cc:492] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:39.425174 4611 tablet_bootstrap.cc:492] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:39.425771 4609 tablet_bootstrap.cc:654] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.426810 4611 tablet_bootstrap.cc:654] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.427243 4613 tablet_bootstrap.cc:492] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:39.427270 4603 tablet_bootstrap.cc:492] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:39.427338 4613 ts_tablet_manager.cc:1367] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 14:00:39.427354 4603 ts_tablet_manager.cc:1367] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20240502 14:00:39.427749 4613 raft_consensus.cc:348] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:39.427884 4613 raft_consensus.cc:374] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.427927 4613 raft_consensus.cc:729] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.427896 4603 raft_consensus.cc:348] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:39.428002 4603 raft_consensus.cc:374] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.428046 4603 raft_consensus.cc:729] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.428042 4613 consensus_queue.cc:260] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:39.428187 4603 consensus_queue.cc:260] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:39.428320 4613 ts_tablet_manager.cc:1398] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:39.428457 4603 ts_tablet_manager.cc:1398] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:39.428614 4603 tablet_bootstrap.cc:492] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:39.429467 4603 tablet_bootstrap.cc:654] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.433038 4609 tablet_bootstrap.cc:492] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:39.433115 4609 ts_tablet_manager.cc:1367] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240502 14:00:39.433307 4603 tablet_bootstrap.cc:492] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:39.433391 4603 ts_tablet_manager.cc:1367] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 14:00:39.433565 4609 raft_consensus.cc:348] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:39.433656 4609 raft_consensus.cc:374] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.433677 4609 raft_consensus.cc:729] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.433797 4603 raft_consensus.cc:348] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:39.433909 4603 raft_consensus.cc:374] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.433944 4603 raft_consensus.cc:729] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.433897 4609 consensus_queue.cc:260] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:39.434051 4603 consensus_queue.cc:260] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:39.434350 4603 ts_tablet_manager.cc:1398] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:39.434849 4609 ts_tablet_manager.cc:1398] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:39.435103 4609 tablet_bootstrap.cc:492] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:39.436093 4609 tablet_bootstrap.cc:654] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:39.437216 4611 tablet_bootstrap.cc:492] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:39.437361 4611 ts_tablet_manager.cc:1367] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.002s
I20240502 14:00:39.437919 4611 raft_consensus.cc:348] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:39.438041 4611 raft_consensus.cc:374] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.438099 4611 raft_consensus.cc:729] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.438237 4611 consensus_queue.cc:260] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:39.438630 4611 ts_tablet_manager.cc:1398] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:39.438724 4609 tablet_bootstrap.cc:492] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:39.438798 4609 ts_tablet_manager.cc:1367] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:39.439361 4609 raft_consensus.cc:348] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:39.439465 4609 raft_consensus.cc:374] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:39.439512 4609 raft_consensus.cc:729] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:39.439620 4609 consensus_queue.cc:260] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:39.440722 4609 ts_tablet_manager.cc:1398] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:39.518306 4553 raft_consensus.cc:482] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:39.518427 4553 raft_consensus.cc:504] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:39.518878 4553 leader_election.cc:288] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:39.519188 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c80435eecda64ecea7864522f5a1dd4c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:39.519341 3151 raft_consensus.cc:2454] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:39.519328 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c80435eecda64ecea7864522f5a1dd4c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:39.519454 541 raft_consensus.cc:2454] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:39.519659 3219 leader_election.cc:302] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:39.519886 4553 raft_consensus.cc:2781] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:39.519945 4553 raft_consensus.cc:482] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:39.519982 4553 raft_consensus.cc:3037] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.520789 4553 raft_consensus.cc:504] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:39.521093 4553 leader_election.cc:288] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:39.521404 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c80435eecda64ecea7864522f5a1dd4c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:39.521485 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c80435eecda64ecea7864522f5a1dd4c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:39.521526 3151 raft_consensus.cc:3037] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.521605 541 raft_consensus.cc:3037] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.522512 3151 raft_consensus.cc:2454] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:39.522512 541 raft_consensus.cc:2454] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:39.522814 3219 leader_election.cc:302] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:39.522979 4553 raft_consensus.cc:2781] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:39.523061 4553 raft_consensus.cc:686] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:39.523226 4553 consensus_queue.cc:237] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:39.525332 32732 catalog_manager.cc:5484] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:39.534204 4553 raft_consensus.cc:482] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:39.534307 4553 raft_consensus.cc:504] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:39.534752 4553 leader_election.cc:288] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:39.535128 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bf7896d6e664a698e9271340c24c908" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:39.535166 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bf7896d6e664a698e9271340c24c908" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:39.535277 672 raft_consensus.cc:2454] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:39.535303 804 raft_consensus.cc:2454] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:39.535606 3219 leader_election.cc:302] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:39.535915 4553 raft_consensus.cc:2781] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:39.535984 4553 raft_consensus.cc:482] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:39.536012 4553 raft_consensus.cc:3037] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.536897 4553 raft_consensus.cc:504] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:39.537266 4553 leader_election.cc:288] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:39.537592 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bf7896d6e664a698e9271340c24c908" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:39.537724 805 raft_consensus.cc:3037] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.538609 805 raft_consensus.cc:2454] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:39.538847 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bf7896d6e664a698e9271340c24c908" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:39.538903 3219 leader_election.cc:302] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:39.538973 672 raft_consensus.cc:3037] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.539050 4553 raft_consensus.cc:2781] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:39.539134 4553 raft_consensus.cc:686] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:39.539266 4553 consensus_queue.cc:237] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:39.540041 672 raft_consensus.cc:2454] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:39.541930 32731 catalog_manager.cc:5484] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:39.574651 4587 raft_consensus.cc:482] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:39.574790 4587 raft_consensus.cc:504] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:39.575273 4587 leader_election.cc:288] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:39.575531 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ffed38329ca4e339a849c5ab3b09e8b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:39.575641 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ffed38329ca4e339a849c5ab3b09e8b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:39.575678 2886 raft_consensus.cc:2454] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:39.575778 3151 raft_consensus.cc:2454] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:39.575991 2954 leader_election.cc:302] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:39.576154 4587 raft_consensus.cc:2781] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:39.576213 4587 raft_consensus.cc:482] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:39.576253 4587 raft_consensus.cc:3037] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.577271 4587 raft_consensus.cc:504] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:39.577610 4587 leader_election.cc:288] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:39.577971 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ffed38329ca4e339a849c5ab3b09e8b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:39.578094 2886 raft_consensus.cc:3037] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.578102 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ffed38329ca4e339a849c5ab3b09e8b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:39.578212 3151 raft_consensus.cc:3037] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.578946 2886 raft_consensus.cc:2454] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:39.579006 3151 raft_consensus.cc:2454] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:39.579216 2954 leader_election.cc:302] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:39.579363 4587 raft_consensus.cc:2781] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:39.579475 4587 raft_consensus.cc:686] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:39.579641 4587 consensus_queue.cc:237] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:39.581961 32731 catalog_manager.cc:5484] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:39.632048 4553 raft_consensus.cc:482] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:39.632165 4553 raft_consensus.cc:504] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:39.632558 4553 leader_election.cc:288] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:39.632856 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c57d60e9354df0930fcc0b3df33f86" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:39.632901 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c57d60e9354df0930fcc0b3df33f86" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:39.632980 672 raft_consensus.cc:2454] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:39.633056 3151 raft_consensus.cc:2454] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:39.633270 3216 leader_election.cc:302] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:39.633440 4553 raft_consensus.cc:2781] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:39.633507 4553 raft_consensus.cc:482] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:39.633560 4553 raft_consensus.cc:3037] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.634424 4553 raft_consensus.cc:504] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:39.634749 4553 leader_election.cc:288] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:39.635043 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c57d60e9354df0930fcc0b3df33f86" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:39.635143 3151 raft_consensus.cc:3037] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.635953 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c57d60e9354df0930fcc0b3df33f86" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:39.635985 3151 raft_consensus.cc:2454] T d9c57d60e9354df0930fcc0b3df33f86 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:39.636055 672 raft_consensus.cc:3037] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:39.636260 3219 leader_election.cc:302] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:39.636417 4553 raft_consensus.cc:2781] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:39.636504 4553 raft_consensus.cc:686] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:39.636643 4553 consensus_queue.cc:237] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:39.637066 672 raft_consensus.cc:2454] T d9c57d60e9354df0930fcc0b3df33f86 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:39.638667 32731 catalog_manager.cc:5484] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:39.679214 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:39.683280 672 raft_consensus.cc:1261] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:39.683336 805 raft_consensus.cc:1261] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:39.683703 4553 consensus_queue.cc:1035] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:39.683902 4370 consensus_queue.cc:1035] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:39.686944 32731 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 8bf7896d6e664a698e9271340c24c908 (table rebalancer_dynamic_table [id=b262cd0ebbae4519adc79bbb9f1f269b]). Expected version 1 got 0
I20240502 14:00:39.687898 32733 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet 8bf7896d6e664a698e9271340c24c908 (table rebalancer_dynamic_table [id=b262cd0ebbae4519adc79bbb9f1f269b]). Expected version 1 got 0
I20240502 14:00:39.688748 4630 tablet.cc:1722] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:39.689318 4630 tablet.cc:1620] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:39.690500 32733 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 8bf7896d6e664a698e9271340c24c908 (table rebalancer_dynamic_table [id=b262cd0ebbae4519adc79bbb9f1f269b]). Expected version 1 got 0
I20240502 14:00:39.693308 32733 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 8bf7896d6e664a698e9271340c24c908 (table rebalancer_dynamic_table [id=b262cd0ebbae4519adc79bbb9f1f269b]). Expected version 1 got 0
I20240502 14:00:39.694653 32732 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet 8bf7896d6e664a698e9271340c24c908 (table rebalancer_dynamic_table [id=b262cd0ebbae4519adc79bbb9f1f269b]). Expected version 1 got 0
I20240502 14:00:39.696180 4632 tablet.cc:1722] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:39.696511 4632 tablet.cc:1620] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:39.696880 4631 tablet.cc:1722] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:39.699299 4631 tablet.cc:1620] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:39.699474 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=b262cd0ebbae4519adc79bbb9f1f269b] alter complete (version 1)
I20240502 14:00:39.801153 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:39.808434 4630 tablet.cc:1722] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:39.808843 4630 tablet.cc:1620] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:39.809926 4631 tablet.cc:1722] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:39.810209 4631 tablet.cc:1620] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:39.812036 4632 tablet.cc:1722] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:39.812357 4632 tablet.cc:1620] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:39.815328 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=b262cd0ebbae4519adc79bbb9f1f269b] alter complete (version 2)
I20240502 14:00:39.918705 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:39.925151 4630 tablet.cc:1722] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:39.925539 4631 tablet.cc:1722] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:39.927275 4632 tablet.cc:1722] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:39.930332 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=b262cd0ebbae4519adc79bbb9f1f269b] alter complete (version 3)
I20240502 14:00:39.934458 4637 consensus_queue.cc:1035] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:39.938032 4553 consensus_queue.cc:1035] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:39.965603 4587 consensus_queue.cc:1035] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:39.968618 4468 consensus_queue.cc:1035] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:40.032678 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:40.032836 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:40.034863 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 8bf7896d6e664a698e9271340c24c908
I20240502 14:00:40.035463 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bf7896d6e664a698e9271340c24c908 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:40 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:40.035480 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bf7896d6e664a698e9271340c24c908 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:40 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:40.035585 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bf7896d6e664a698e9271340c24c908 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:40 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:40.036312 4649 tablet_replica.cc:330] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:40.036541 4649 raft_consensus.cc:2229] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:40.036710 4649 raft_consensus.cc:2258] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:40.037397 4649 ts_tablet_manager.cc:1875] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:40.039944 4649 ts_tablet_manager.cc:1888] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:40.040089 4649 log.cc:1196] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/8bf7896d6e664a698e9271340c24c908
I20240502 14:00:40.040437 4649 ts_tablet_manager.cc:1909] T 8bf7896d6e664a698e9271340c24c908 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:40.041219 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 8bf7896d6e664a698e9271340c24c908 (table rebalancer_dynamic_table_new_name [id=b262cd0ebbae4519adc79bbb9f1f269b]) successfully deleted
I20240502 14:00:40.041458 4648 tablet_replica.cc:330] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:40.041632 4648 raft_consensus.cc:2229] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:40.041782 4648 raft_consensus.cc:2258] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:40.042053 4647 tablet_replica.cc:330] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:40.042217 4647 raft_consensus.cc:2229] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:40.042418 4647 raft_consensus.cc:2258] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:40.042475 4648 ts_tablet_manager.cc:1875] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:40.042871 4647 ts_tablet_manager.cc:1875] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:40.044752 4647 ts_tablet_manager.cc:1888] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:40.044859 4647 log.cc:1196] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/8bf7896d6e664a698e9271340c24c908
I20240502 14:00:40.044838 4648 ts_tablet_manager.cc:1888] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:40.044987 4648 log.cc:1196] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/8bf7896d6e664a698e9271340c24c908
I20240502 14:00:40.045202 4647 ts_tablet_manager.cc:1909] T 8bf7896d6e664a698e9271340c24c908 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:40.045387 4648 ts_tablet_manager.cc:1909] T 8bf7896d6e664a698e9271340c24c908 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:40.045881 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 8bf7896d6e664a698e9271340c24c908 (table rebalancer_dynamic_table_new_name [id=b262cd0ebbae4519adc79bbb9f1f269b]) successfully deleted
I20240502 14:00:40.046082 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 8bf7896d6e664a698e9271340c24c908 (table rebalancer_dynamic_table_new_name [id=b262cd0ebbae4519adc79bbb9f1f269b]) successfully deleted
I20240502 14:00:40.118762 4553 consensus_queue.cc:1035] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:40.121814 4370 consensus_queue.cc:1035] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:40.136749 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:40.141651 2866 tablet_service.cc:1467] Processing CreateTablet for tablet d78082b4cdb143b1808ab4f507608ad7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=490906fa01f442d3a5a534f84a8eddd6]), partition=
I20240502 14:00:40.141685 522 tablet_service.cc:1467] Processing CreateTablet for tablet d78082b4cdb143b1808ab4f507608ad7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=490906fa01f442d3a5a534f84a8eddd6]), partition=
I20240502 14:00:40.141934 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d78082b4cdb143b1808ab4f507608ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.141934 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d78082b4cdb143b1808ab4f507608ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.141921 3132 tablet_service.cc:1467] Processing CreateTablet for tablet d78082b4cdb143b1808ab4f507608ad7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=490906fa01f442d3a5a534f84a8eddd6]), partition=
I20240502 14:00:40.142138 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d78082b4cdb143b1808ab4f507608ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.143971 4652 tablet_bootstrap.cc:492] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:40.144335 4654 tablet_bootstrap.cc:492] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:40.144379 4653 tablet_bootstrap.cc:492] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:40.144958 4652 tablet_bootstrap.cc:654] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.145349 4653 tablet_bootstrap.cc:654] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.145362 4654 tablet_bootstrap.cc:654] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.147501 4653 tablet_bootstrap.cc:492] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:40.147644 4653 ts_tablet_manager.cc:1367] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:40.147910 4652 tablet_bootstrap.cc:492] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:40.148010 4652 ts_tablet_manager.cc:1367] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:40.148223 4653 raft_consensus.cc:348] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.148326 4653 raft_consensus.cc:374] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.148360 4653 raft_consensus.cc:729] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.148487 4653 consensus_queue.cc:260] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.148463 4652 raft_consensus.cc:348] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.148576 4652 raft_consensus.cc:374] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.148617 4652 raft_consensus.cc:729] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.148738 4652 consensus_queue.cc:260] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.148792 4653 ts_tablet_manager.cc:1398] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:40.149024 4652 ts_tablet_manager.cc:1398] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:40.149801 4654 tablet_bootstrap.cc:492] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:40.149895 4654 ts_tablet_manager.cc:1367] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:40.150318 4654 raft_consensus.cc:348] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.150609 4654 raft_consensus.cc:374] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.150671 4654 raft_consensus.cc:729] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.151175 4654 consensus_queue.cc:260] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.154815 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:40.156584 4654 ts_tablet_manager.cc:1398] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:00:40.161451 388 tablet_service.cc:1467] Processing CreateTablet for tablet 564ba4de769e45af8de10fe8c86eef31 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=70b6fe08fece4cc7b481dfd7c0360f3e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:40.161450 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 564ba4de769e45af8de10fe8c86eef31 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=70b6fe08fece4cc7b481dfd7c0360f3e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:40.161562 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 564ba4de769e45af8de10fe8c86eef31 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=70b6fe08fece4cc7b481dfd7c0360f3e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:40.161626 386 tablet_service.cc:1467] Processing CreateTablet for tablet 604579a2517d4c2eaf855db299d15d5e (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=70b6fe08fece4cc7b481dfd7c0360f3e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:40.161739 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 564ba4de769e45af8de10fe8c86eef31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.161742 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 564ba4de769e45af8de10fe8c86eef31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.161777 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 564ba4de769e45af8de10fe8c86eef31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.161996 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 604579a2517d4c2eaf855db299d15d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.164129 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 604579a2517d4c2eaf855db299d15d5e (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=70b6fe08fece4cc7b481dfd7c0360f3e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:40.164352 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 604579a2517d4c2eaf855db299d15d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.165007 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 604579a2517d4c2eaf855db299d15d5e (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=70b6fe08fece4cc7b481dfd7c0360f3e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:40.165293 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 604579a2517d4c2eaf855db299d15d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.165391 4660 tablet_bootstrap.cc:492] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:40.166182 4660 tablet_bootstrap.cc:654] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.167021 4659 tablet_bootstrap.cc:492] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:40.167598 4662 tablet_bootstrap.cc:492] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:40.167666 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 420dafd799694900b43d5df9cf4ed483 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=70b6fe08fece4cc7b481dfd7c0360f3e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:40.167726 784 tablet_service.cc:1467] Processing CreateTablet for tablet 420dafd799694900b43d5df9cf4ed483 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=70b6fe08fece4cc7b481dfd7c0360f3e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:40.167913 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420dafd799694900b43d5df9cf4ed483. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.168010 4659 tablet_bootstrap.cc:654] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.168017 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420dafd799694900b43d5df9cf4ed483. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.168588 4662 tablet_bootstrap.cc:654] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.169044 4659 tablet_bootstrap.cc:492] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:40.169129 4659 ts_tablet_manager.cc:1367] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:40.169610 4659 raft_consensus.cc:348] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.169724 4659 raft_consensus.cc:374] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.169757 4659 raft_consensus.cc:729] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.169890 4659 consensus_queue.cc:260] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.170183 4659 ts_tablet_manager.cc:1398] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:40.170357 4665 tablet_bootstrap.cc:492] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:40.170944 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 420dafd799694900b43d5df9cf4ed483 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=70b6fe08fece4cc7b481dfd7c0360f3e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:40.171192 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420dafd799694900b43d5df9cf4ed483. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:40.171411 4665 tablet_bootstrap.cc:654] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.172571 4652 tablet_bootstrap.cc:492] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:40.172909 4662 tablet_bootstrap.cc:492] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:40.172986 4662 ts_tablet_manager.cc:1367] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:40.173398 4662 raft_consensus.cc:348] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.173545 4652 tablet_bootstrap.cc:654] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.173591 4662 raft_consensus.cc:374] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.173643 4662 raft_consensus.cc:729] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.173774 4662 consensus_queue.cc:260] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.174134 4662 ts_tablet_manager.cc:1398] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:40.174458 4662 tablet_bootstrap.cc:492] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:40.175361 4652 tablet_bootstrap.cc:492] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:40.175487 4662 tablet_bootstrap.cc:654] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.175747 4652 ts_tablet_manager.cc:1367] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:40.175757 4660 tablet_bootstrap.cc:492] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:40.175873 4660 ts_tablet_manager.cc:1367] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:00:40.176440 4660 raft_consensus.cc:348] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.176560 4660 raft_consensus.cc:374] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.176640 4660 raft_consensus.cc:729] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.176776 4660 consensus_queue.cc:260] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.177151 4660 ts_tablet_manager.cc:1398] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:40.176395 4652 raft_consensus.cc:348] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.177556 4660 tablet_bootstrap.cc:492] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:40.177558 4652 raft_consensus.cc:374] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.177605 4652 raft_consensus.cc:729] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.177824 4665 tablet_bootstrap.cc:492] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:40.177904 4665 ts_tablet_manager.cc:1367] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:40.178378 4665 raft_consensus.cc:348] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:40.178493 4665 raft_consensus.cc:374] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.178480 4652 consensus_queue.cc:260] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.178531 4665 raft_consensus.cc:729] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.178680 4660 tablet_bootstrap.cc:654] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.178676 4665 consensus_queue.cc:260] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:40.179263 4665 ts_tablet_manager.cc:1398] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:40.180124 4662 tablet_bootstrap.cc:492] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:40.180236 4662 ts_tablet_manager.cc:1367] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:40.180490 4652 ts_tablet_manager.cc:1398] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:40.180727 4662 raft_consensus.cc:348] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:40.180851 4662 raft_consensus.cc:374] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.180922 4662 raft_consensus.cc:729] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.181088 4662 consensus_queue.cc:260] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:40.181353 4652 tablet_bootstrap.cc:492] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:40.182089 4662 ts_tablet_manager.cc:1398] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:40.182937 4652 tablet_bootstrap.cc:654] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.183593 4660 tablet_bootstrap.cc:492] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:40.183744 4660 ts_tablet_manager.cc:1367] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:40.184091 4652 tablet_bootstrap.cc:492] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:40.184162 4652 ts_tablet_manager.cc:1367] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:40.184408 4660 raft_consensus.cc:348] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.184504 4660 raft_consensus.cc:374] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.184535 4660 raft_consensus.cc:729] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.184589 4652 raft_consensus.cc:348] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.184684 4652 raft_consensus.cc:374] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.184716 4652 raft_consensus.cc:729] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.184677 4660 consensus_queue.cc:260] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.184823 4652 consensus_queue.cc:260] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.185110 4652 ts_tablet_manager.cc:1398] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:40.185128 4660 ts_tablet_manager.cc:1398] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:40.185267 4652 tablet_bootstrap.cc:492] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:40.186260 4652 tablet_bootstrap.cc:654] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:40.188513 4652 tablet_bootstrap.cc:492] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:40.189504 4652 ts_tablet_manager.cc:1367] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:40.190393 4652 raft_consensus.cc:348] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:40.190569 4652 raft_consensus.cc:374] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:40.190616 4652 raft_consensus.cc:729] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:40.190737 4652 consensus_queue.cc:260] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:40.191018 4652 ts_tablet_manager.cc:1398] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 14:00:40.299511 589 tablet.cc:2369] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:40.349241 4634 raft_consensus.cc:482] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:40.349404 4634 raft_consensus.cc:504] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:40.350142 4634 leader_election.cc:288] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:40.350589 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420dafd799694900b43d5df9cf4ed483" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:40.350785 3151 raft_consensus.cc:2454] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:40.351121 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420dafd799694900b43d5df9cf4ed483" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:40.351258 741 leader_election.cc:302] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:40.351274 3019 raft_consensus.cc:2454] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:40.351996 4634 raft_consensus.cc:2781] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:40.352102 4634 raft_consensus.cc:482] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:40.352161 4634 raft_consensus.cc:3037] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.353271 4634 raft_consensus.cc:504] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:40.353843 4634 leader_election.cc:288] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:40.356446 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420dafd799694900b43d5df9cf4ed483" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:40.356496 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420dafd799694900b43d5df9cf4ed483" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:40.356635 3151 raft_consensus.cc:3037] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.356663 3019 raft_consensus.cc:3037] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.357419 3019 raft_consensus.cc:2454] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:40.357797 738 leader_election.cc:302] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:40.357851 3151 raft_consensus.cc:2454] T 420dafd799694900b43d5df9cf4ed483 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:40.357980 4634 raft_consensus.cc:2781] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:40.358134 4634 raft_consensus.cc:686] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:40.358289 4634 consensus_queue.cc:237] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
W20240502 14:00:40.361613 457 tablet.cc:2369] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:40.362568 32726 catalog_manager.cc:5484] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } }
W20240502 14:00:40.417497 853 tablet.cc:2369] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:40.461510 4194 raft_consensus.cc:482] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:40.461625 4194 raft_consensus.cc:504] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.462065 4194 leader_election.cc:288] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:40.462447 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "564ba4de769e45af8de10fe8c86eef31" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:40.462587 3283 raft_consensus.cc:2454] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:40.462553 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "564ba4de769e45af8de10fe8c86eef31" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:40.462679 409 raft_consensus.cc:2454] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:40.463035 3086 leader_election.cc:302] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:40.463241 4194 raft_consensus.cc:2781] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:40.463307 4194 raft_consensus.cc:482] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:40.463363 4194 raft_consensus.cc:3037] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.464241 4194 raft_consensus.cc:504] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.464551 4194 leader_election.cc:288] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:40.464911 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "564ba4de769e45af8de10fe8c86eef31" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:40.465046 3283 raft_consensus.cc:3037] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.465042 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "564ba4de769e45af8de10fe8c86eef31" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:40.465135 409 raft_consensus.cc:3037] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.465831 3283 raft_consensus.cc:2454] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:40.465867 409 raft_consensus.cc:2454] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:40.466133 3086 leader_election.cc:302] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:40.466285 4194 raft_consensus.cc:2781] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:40.466645 4194 raft_consensus.cc:686] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:40.466785 4194 consensus_queue.cc:237] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.468796 32726 catalog_manager.cc:5484] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:40.504700 4633 raft_consensus.cc:482] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:40.504829 4633 raft_consensus.cc:504] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.505344 4633 leader_election.cc:288] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:40.505898 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "604579a2517d4c2eaf855db299d15d5e" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:40.506043 3019 raft_consensus.cc:2454] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:40.506095 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "604579a2517d4c2eaf855db299d15d5e" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:40.506253 3151 raft_consensus.cc:2454] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:40.506356 341 leader_election.cc:302] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:40.506521 4633 raft_consensus.cc:2781] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:40.506579 4633 raft_consensus.cc:482] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:40.506629 4633 raft_consensus.cc:3037] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.507658 4633 raft_consensus.cc:504] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.508028 4633 leader_election.cc:288] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:40.508276 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "604579a2517d4c2eaf855db299d15d5e" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:40.508396 3019 raft_consensus.cc:3037] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.508421 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "604579a2517d4c2eaf855db299d15d5e" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:40.508841 3151 raft_consensus.cc:3037] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.509277 3019 raft_consensus.cc:2454] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:40.509557 341 leader_election.cc:302] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:40.509770 4633 raft_consensus.cc:2781] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:40.509860 4633 raft_consensus.cc:686] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:40.509876 3151 raft_consensus.cc:2454] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:40.510004 4633 consensus_queue.cc:237] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.512089 32726 catalog_manager.cc:5484] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:40.558285 4636 raft_consensus.cc:482] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:40.558455 4636 raft_consensus.cc:504] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.559288 4636 leader_election.cc:288] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:40.559690 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d78082b4cdb143b1808ab4f507608ad7" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:40.559919 3151 raft_consensus.cc:2454] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:40.559960 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d78082b4cdb143b1808ab4f507608ad7" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:40.560089 2884 raft_consensus.cc:2454] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:40.560362 477 leader_election.cc:302] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:40.560583 4636 raft_consensus.cc:2781] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:40.560658 4636 raft_consensus.cc:482] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:40.560706 4636 raft_consensus.cc:3037] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.561846 4636 raft_consensus.cc:504] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.562114 4636 leader_election.cc:288] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:40.562472 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d78082b4cdb143b1808ab4f507608ad7" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:40.562479 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d78082b4cdb143b1808ab4f507608ad7" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:40.562589 3151 raft_consensus.cc:3037] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.562590 2884 raft_consensus.cc:3037] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:40.563563 2884 raft_consensus.cc:2454] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:40.563580 3151 raft_consensus.cc:2454] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:40.563875 477 leader_election.cc:302] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:40.564162 4636 raft_consensus.cc:2781] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:40.564257 4636 raft_consensus.cc:686] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:40.564392 4636 consensus_queue.cc:237] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:40.566435 32726 catalog_manager.cc:5484] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:40.726256 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:40.730686 3151 raft_consensus.cc:1261] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:40.730778 2884 raft_consensus.cc:1261] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:40.732647 4541 consensus_queue.cc:1035] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:40.735131 4541 consensus_queue.cc:1035] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:40.739001 4681 tablet.cc:1722] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:40.739724 4681 tablet.cc:1620] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:40.744964 32726 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet d78082b4cdb143b1808ab4f507608ad7 (table rebalancer_dynamic_table [id=490906fa01f442d3a5a534f84a8eddd6]). Expected version 1 got 0
I20240502 14:00:40.747165 32732 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet d78082b4cdb143b1808ab4f507608ad7 (table rebalancer_dynamic_table [id=490906fa01f442d3a5a534f84a8eddd6]). Expected version 1 got 0
I20240502 14:00:40.747545 32726 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet d78082b4cdb143b1808ab4f507608ad7 (table rebalancer_dynamic_table [id=490906fa01f442d3a5a534f84a8eddd6]). Expected version 1 got 0
I20240502 14:00:40.751247 32732 catalog_manager.cc:5576] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751) does not have the latest schema for tablet d78082b4cdb143b1808ab4f507608ad7 (table rebalancer_dynamic_table [id=490906fa01f442d3a5a534f84a8eddd6]). Expected version 1 got 0
I20240502 14:00:40.752671 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=490906fa01f442d3a5a534f84a8eddd6] alter complete (version 1)
I20240502 14:00:40.757472 32731 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet d78082b4cdb143b1808ab4f507608ad7 (table rebalancer_dynamic_table [id=490906fa01f442d3a5a534f84a8eddd6]). Expected version 1 got 0
I20240502 14:00:40.758373 4685 tablet.cc:1722] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:40.758810 4685 tablet.cc:1620] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:40.761265 4686 tablet.cc:1722] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:40.762897 4686 tablet.cc:1620] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:40.858599 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:40.867339 4681 tablet.cc:1722] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:40.867516 4685 tablet.cc:1722] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:40.867697 4681 tablet.cc:1620] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:40.867904 4685 tablet.cc:1620] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:40.872742 4686 tablet.cc:1722] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:40.873221 4686 tablet.cc:1620] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: MemRowSet was empty: no flush needed.
I20240502 14:00:40.873575 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=490906fa01f442d3a5a534f84a8eddd6] alter complete (version 2)
I20240502 14:00:40.880635 4634 consensus_queue.cc:1035] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:40.884709 4690 consensus_queue.cc:1035] T 420dafd799694900b43d5df9cf4ed483 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:40.930421 4633 consensus_queue.cc:1035] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:40.933822 4695 consensus_queue.cc:1035] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:40.968271 4598 consensus_queue.cc:1035] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:40.972146 4698 consensus_queue.cc:1035] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:40.978796 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:40.984997 4681 tablet.cc:1722] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:40.985749 4685 tablet.cc:1722] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:40.989308 4686 tablet.cc:1722] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:40.990343 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=490906fa01f442d3a5a534f84a8eddd6] alter complete (version 3)
I20240502 14:00:41.039091 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:41.046574 784 tablet_service.cc:1467] Processing CreateTablet for tablet 56a192eba0af4486a212972ffdfacf9d (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.046921 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56a192eba0af4486a212972ffdfacf9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.048375 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 6423e05535aa41d5923f570c7b73f61c (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.048426 2864 tablet_service.cc:1467] Processing CreateTablet for tablet d493cec70a39401b98c21446f866789a (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.048672 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6423e05535aa41d5923f570c7b73f61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.048632 653 tablet_service.cc:1467] Processing CreateTablet for tablet 56a192eba0af4486a212972ffdfacf9d (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.048879 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56a192eba0af4486a212972ffdfacf9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.049305 4701 tablet_bootstrap.cc:492] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:41.050626 4701 tablet_bootstrap.cc:654] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.051359 654 tablet_service.cc:1467] Processing CreateTablet for tablet 6423e05535aa41d5923f570c7b73f61c (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.051612 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6423e05535aa41d5923f570c7b73f61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.051888 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d493cec70a39401b98c21446f866789a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.052027 4701 tablet_bootstrap.cc:492] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:41.052141 4701 ts_tablet_manager.cc:1367] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:41.052521 2998 tablet_service.cc:1467] Processing CreateTablet for tablet d493cec70a39401b98c21446f866789a (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.052812 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d493cec70a39401b98c21446f866789a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.053303 4704 tablet_bootstrap.cc:492] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:41.053890 3263 tablet_service.cc:1467] Processing CreateTablet for tablet d493cec70a39401b98c21446f866789a (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.054177 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d493cec70a39401b98c21446f866789a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.054147 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 56a192eba0af4486a212972ffdfacf9d (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.054360 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56a192eba0af4486a212972ffdfacf9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.054702 4704 tablet_bootstrap.cc:654] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.055657 4701 raft_consensus.cc:348] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:41.055845 4701 raft_consensus.cc:374] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.055909 4701 raft_consensus.cc:729] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.056056 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 6423e05535aa41d5923f570c7b73f61c (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.056066 4701 consensus_queue.cc:260] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:41.056231 4704 tablet_bootstrap.cc:492] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:41.056288 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6423e05535aa41d5923f570c7b73f61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.056304 4704 ts_tablet_manager.cc:1367] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:41.056381 4701 ts_tablet_manager.cc:1398] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:41.056725 4704 raft_consensus.cc:348] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.056839 4704 raft_consensus.cc:374] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.056880 4704 raft_consensus.cc:729] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.057019 4704 consensus_queue.cc:260] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.057283 4704 ts_tablet_manager.cc:1398] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:41.057437 4704 tablet_bootstrap.cc:492] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:41.058250 4707 tablet_bootstrap.cc:492] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:41.058302 4704 tablet_bootstrap.cc:654] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.059228 4707 tablet_bootstrap.cc:654] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.059442 4705 tablet_bootstrap.cc:492] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:41.059682 4702 tablet_bootstrap.cc:492] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:41.060675 4702 tablet_bootstrap.cc:654] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.061149 4705 tablet_bootstrap.cc:654] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.062039 4702 tablet_bootstrap.cc:492] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:41.062142 4702 ts_tablet_manager.cc:1367] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:41.062253 4707 tablet_bootstrap.cc:492] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:41.062335 4707 ts_tablet_manager.cc:1367] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:41.062614 4702 raft_consensus.cc:348] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:41.062737 4702 raft_consensus.cc:374] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.062767 4702 raft_consensus.cc:729] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.062876 4702 consensus_queue.cc:260] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:41.063145 4702 ts_tablet_manager.cc:1398] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:41.063294 4702 tablet_bootstrap.cc:492] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:41.064018 4707 raft_consensus.cc:348] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.064160 4707 raft_consensus.cc:374] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.064188 4704 tablet_bootstrap.cc:492] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:41.064214 4707 raft_consensus.cc:729] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.064364 4707 consensus_queue.cc:260] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.064410 4704 ts_tablet_manager.cc:1367] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 14:00:41.064669 4707 ts_tablet_manager.cc:1398] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:41.066640 4702 tablet_bootstrap.cc:654] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.067276 4704 raft_consensus.cc:348] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.067392 4704 raft_consensus.cc:374] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.067432 4704 raft_consensus.cc:729] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.067607 4704 consensus_queue.cc:260] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.069250 4704 ts_tablet_manager.cc:1398] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 14:00:41.069473 4705 tablet_bootstrap.cc:492] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:41.069592 4705 ts_tablet_manager.cc:1367] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:41.069938 4702 tablet_bootstrap.cc:492] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:41.070029 4702 ts_tablet_manager.cc:1367] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:41.070108 4705 raft_consensus.cc:348] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.070215 4705 raft_consensus.cc:374] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.070246 4705 raft_consensus.cc:729] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.070359 4705 consensus_queue.cc:260] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.070457 4702 raft_consensus.cc:348] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.070565 4702 raft_consensus.cc:374] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.070601 4702 raft_consensus.cc:729] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.070715 4705 ts_tablet_manager.cc:1398] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:41.070722 4702 consensus_queue.cc:260] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.070981 4705 tablet_bootstrap.cc:492] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:41.070991 4702 ts_tablet_manager.cc:1398] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:41.072054 4705 tablet_bootstrap.cc:654] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.073334 4705 tablet_bootstrap.cc:492] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:41.073477 4705 ts_tablet_manager.cc:1367] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:41.074088 4705 raft_consensus.cc:348] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.074251 4705 raft_consensus.cc:374] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.074324 4705 raft_consensus.cc:729] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.074479 4705 consensus_queue.cc:260] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.074867 4705 ts_tablet_manager.cc:1398] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:41.075048 4705 tablet_bootstrap.cc:492] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:41.076030 4705 tablet_bootstrap.cc:654] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.077035 4705 tablet_bootstrap.cc:492] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:41.077118 4705 ts_tablet_manager.cc:1367] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:41.077687 4705 raft_consensus.cc:348] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:41.077808 4705 raft_consensus.cc:374] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.077847 4705 raft_consensus.cc:729] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.077965 4705 consensus_queue.cc:260] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:41.078292 4705 ts_tablet_manager.cc:1398] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:41.092541 32726 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:41.092679 32726 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:41.094630 32726 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet d78082b4cdb143b1808ab4f507608ad7
I20240502 14:00:41.095283 522 tablet_service.cc:1514] Processing DeleteTablet for tablet d78082b4cdb143b1808ab4f507608ad7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:41 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:41.095348 2864 tablet_service.cc:1514] Processing DeleteTablet for tablet d78082b4cdb143b1808ab4f507608ad7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:41 UTC) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:41.095655 4715 tablet_replica.cc:330] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:41.095674 4716 tablet_replica.cc:330] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:41.095654 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet d78082b4cdb143b1808ab4f507608ad7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:41 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:41.095839 4715 raft_consensus.cc:2229] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:41.095851 4716 raft_consensus.cc:2229] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:41.095997 4717 tablet_replica.cc:330] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:41.096041 4716 raft_consensus.cc:2258] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:41.096180 4717 raft_consensus.cc:2229] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:41.096324 4717 raft_consensus.cc:2258] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:41.096437 4715 raft_consensus.cc:2258] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:41.097117 4716 ts_tablet_manager.cc:1875] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:41.097136 4717 ts_tablet_manager.cc:1875] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:41.097245 4715 ts_tablet_manager.cc:1875] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:41.099346 4717 ts_tablet_manager.cc:1888] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:41.099486 4717 log.cc:1196] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/d78082b4cdb143b1808ab4f507608ad7
I20240502 14:00:41.099514 4716 ts_tablet_manager.cc:1888] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:41.099596 4716 log.cc:1196] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/d78082b4cdb143b1808ab4f507608ad7
I20240502 14:00:41.099931 4716 ts_tablet_manager.cc:1909] T d78082b4cdb143b1808ab4f507608ad7 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting consensus metadata
I20240502 14:00:41.099957 4717 ts_tablet_manager.cc:1909] T d78082b4cdb143b1808ab4f507608ad7 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:41.100368 4715 ts_tablet_manager.cc:1888] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:41.100481 4715 log.cc:1196] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/d78082b4cdb143b1808ab4f507608ad7
I20240502 14:00:41.100653 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet d78082b4cdb143b1808ab4f507608ad7 (table rebalancer_dynamic_table_new_name [id=490906fa01f442d3a5a534f84a8eddd6]) successfully deleted
I20240502 14:00:41.100837 4715 ts_tablet_manager.cc:1909] T d78082b4cdb143b1808ab4f507608ad7 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:41.100831 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet d78082b4cdb143b1808ab4f507608ad7 (table rebalancer_dynamic_table_new_name [id=490906fa01f442d3a5a534f84a8eddd6]) successfully deleted
I20240502 14:00:41.101482 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet d78082b4cdb143b1808ab4f507608ad7 (table rebalancer_dynamic_table_new_name [id=490906fa01f442d3a5a534f84a8eddd6]) successfully deleted
I20240502 14:00:41.139511 4690 raft_consensus.cc:482] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:41.139647 4690 raft_consensus.cc:504] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:41.140156 4690 leader_election.cc:288] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:41.140444 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a192eba0af4486a212972ffdfacf9d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:41.140602 3019 raft_consensus.cc:2454] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:41.140581 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a192eba0af4486a212972ffdfacf9d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:41.140707 672 raft_consensus.cc:2454] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:41.140918 738 leader_election.cc:302] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:41.141237 4690 raft_consensus.cc:2781] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:41.141310 4690 raft_consensus.cc:482] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:41.141362 4690 raft_consensus.cc:3037] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.142315 4690 raft_consensus.cc:504] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:41.142628 4690 leader_election.cc:288] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:41.142931 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a192eba0af4486a212972ffdfacf9d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:41.143049 3019 raft_consensus.cc:3037] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.143028 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a192eba0af4486a212972ffdfacf9d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:41.143172 672 raft_consensus.cc:3037] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.143805 3019 raft_consensus.cc:2454] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:41.143978 672 raft_consensus.cc:2454] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:41.144191 738 leader_election.cc:302] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:41.144483 4690 raft_consensus.cc:2781] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:41.144567 4690 raft_consensus.cc:686] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:41.144714 4690 consensus_queue.cc:237] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:41.146946 32726 catalog_manager.cc:5484] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:41.195765 4676 raft_consensus.cc:482] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:41.195923 4676 raft_consensus.cc:504] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.196208 32726 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:41.196359 4676 leader_election.cc:288] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:41.196691 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6423e05535aa41d5923f570c7b73f61c" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:41.196837 2884 raft_consensus.cc:2454] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:41.197273 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6423e05535aa41d5923f570c7b73f61c" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:41.197407 672 raft_consensus.cc:2454] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:41.197768 2951 leader_election.cc:302] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:41.197999 4676 raft_consensus.cc:2781] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:41.198081 4676 raft_consensus.cc:482] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:41.198128 4676 raft_consensus.cc:3037] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.199321 4676 raft_consensus.cc:504] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.199658 4676 leader_election.cc:288] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:41.200206 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6423e05535aa41d5923f570c7b73f61c" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:41.200320 672 raft_consensus.cc:3037] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.200414 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6423e05535aa41d5923f570c7b73f61c" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:41.200534 2884 raft_consensus.cc:3037] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.201365 672 raft_consensus.cc:2454] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:41.201530 2884 raft_consensus.cc:2454] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:41.202049 2954 leader_election.cc:302] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:41.202221 4676 raft_consensus.cc:2781] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:41.202598 4676 raft_consensus.cc:686] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:41.202773 4718 raft_consensus.cc:482] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:41.202741 4676 consensus_queue.cc:237] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.202850 4718 raft_consensus.cc:504] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.203388 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 80611ba2ee4a495185189c8874c0529b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=af5660f0891c4705a5ec3823b7b2dc33]), partition=
I20240502 14:00:41.203636 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80611ba2ee4a495185189c8874c0529b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.203763 784 tablet_service.cc:1467] Processing CreateTablet for tablet 80611ba2ee4a495185189c8874c0529b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=af5660f0891c4705a5ec3823b7b2dc33]), partition=
I20240502 14:00:41.204044 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80611ba2ee4a495185189c8874c0529b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.204511 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d493cec70a39401b98c21446f866789a" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:41.204613 3019 raft_consensus.cc:2454] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:41.204867 2819 leader_election.cc:302] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:41.205041 4657 raft_consensus.cc:2781] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:41.205099 4657 raft_consensus.cc:482] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:41.205140 4657 raft_consensus.cc:3037] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.205044 4718 leader_election.cc:288] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:41.206079 4657 raft_consensus.cc:504] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.206333 4721 tablet_bootstrap.cc:492] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:41.206380 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 80611ba2ee4a495185189c8874c0529b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=af5660f0891c4705a5ec3823b7b2dc33]), partition=
I20240502 14:00:41.206408 4657 leader_election.cc:288] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:41.206579 4701 tablet_bootstrap.cc:492] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:41.206599 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80611ba2ee4a495185189c8874c0529b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.207098 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d493cec70a39401b98c21446f866789a" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:41.207233 3283 raft_consensus.cc:2454] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:41.207334 4721 tablet_bootstrap.cc:654] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.207466 4701 tablet_bootstrap.cc:654] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.208089 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d493cec70a39401b98c21446f866789a" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:41.208182 3283 raft_consensus.cc:3037] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.208990 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d493cec70a39401b98c21446f866789a" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:41.209219 3283 raft_consensus.cc:2454] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:41.209347 4721 tablet_bootstrap.cc:492] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:41.209360 3019 raft_consensus.cc:3037] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.209470 4721 ts_tablet_manager.cc:1367] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:41.209524 2820 leader_election.cc:302] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:41.209705 4657 raft_consensus.cc:2781] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:41.209789 4657 raft_consensus.cc:686] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:41.209767 32726 catalog_manager.cc:5484] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:41.209914 4657 consensus_queue.cc:237] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.209986 4721 raft_consensus.cc:348] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:41.210109 4721 raft_consensus.cc:374] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.210167 4721 raft_consensus.cc:729] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.210295 4721 consensus_queue.cc:260] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:41.210579 4721 ts_tablet_manager.cc:1398] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:41.211306 4701 tablet_bootstrap.cc:492] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:41.211395 4701 ts_tablet_manager.cc:1367] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:41.211896 4701 raft_consensus.cc:348] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:41.212011 4701 raft_consensus.cc:374] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.212047 4701 raft_consensus.cc:729] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.212154 4701 consensus_queue.cc:260] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:41.212450 4701 ts_tablet_manager.cc:1398] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:41.212656 32726 catalog_manager.cc:5484] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:41.228756 3019 raft_consensus.cc:2454] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:41.229782 4705 tablet_bootstrap.cc:492] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:41.230801 4705 tablet_bootstrap.cc:654] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.231864 4705 tablet_bootstrap.cc:492] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:41.231952 4705 ts_tablet_manager.cc:1367] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:41.232309 4705 raft_consensus.cc:348] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:41.232425 4705 raft_consensus.cc:374] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.232481 4705 raft_consensus.cc:729] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.232614 4705 consensus_queue.cc:260] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:41.232903 4705 ts_tablet_manager.cc:1398] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 14:00:41.248425 721 tablet.cc:2369] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:41.490830 4676 raft_consensus.cc:482] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:41.490950 4676 raft_consensus.cc:504] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:41.491387 4676 leader_election.cc:288] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:41.491726 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80611ba2ee4a495185189c8874c0529b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:41.491801 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80611ba2ee4a495185189c8874c0529b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:41.491899 3151 raft_consensus.cc:2454] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:41.491963 805 raft_consensus.cc:2454] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:41.492204 2954 leader_election.cc:302] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:41.492409 4676 raft_consensus.cc:2781] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:41.492470 4676 raft_consensus.cc:482] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:41.492497 4676 raft_consensus.cc:3037] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.493458 4676 raft_consensus.cc:504] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:41.493762 4676 leader_election.cc:288] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:41.494096 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80611ba2ee4a495185189c8874c0529b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:41.494122 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80611ba2ee4a495185189c8874c0529b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:41.494215 3151 raft_consensus.cc:3037] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.494236 805 raft_consensus.cc:3037] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:41.495237 805 raft_consensus.cc:2454] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:41.495507 2954 leader_election.cc:302] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:41.495730 4676 raft_consensus.cc:2781] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:41.495770 3151 raft_consensus.cc:2454] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:41.495810 4676 raft_consensus.cc:686] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:41.495967 4676 consensus_queue.cc:237] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:41.498010 32726 catalog_manager.cc:5484] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:41.565661 4726 consensus_queue.cc:1035] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:41.569761 4690 consensus_queue.cc:1035] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:41.616488 4718 consensus_queue.cc:1035] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:41.620605 4657 consensus_queue.cc:1035] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:41.653443 4738 consensus_queue.cc:1035] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:41.657078 4676 consensus_queue.cc:1035] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:41.666961 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:41.670995 3150 raft_consensus.cc:1261] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:41.671118 804 raft_consensus.cc:1261] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:41.671669 4738 consensus_queue.cc:1035] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:41.671842 4676 consensus_queue.cc:1035] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:41.674691 32733 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet 80611ba2ee4a495185189c8874c0529b (table rebalancer_dynamic_table [id=af5660f0891c4705a5ec3823b7b2dc33]). Expected version 1 got 0
I20240502 14:00:41.675047 32726 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 80611ba2ee4a495185189c8874c0529b (table rebalancer_dynamic_table [id=af5660f0891c4705a5ec3823b7b2dc33]). Expected version 1 got 0
I20240502 14:00:41.676694 4744 tablet.cc:1722] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:41.677044 4744 tablet.cc:1620] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:41.680646 4745 tablet.cc:1722] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:41.680958 4745 tablet.cc:1620] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:41.681277 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=af5660f0891c4705a5ec3823b7b2dc33] alter complete (version 1)
I20240502 14:00:41.684495 4746 tablet.cc:1722] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:41.684702 32731 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 80611ba2ee4a495185189c8874c0529b (table rebalancer_dynamic_table [id=af5660f0891c4705a5ec3823b7b2dc33]). Expected version 1 got 0
I20240502 14:00:41.684824 4746 tablet.cc:1620] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:41.750259 32726 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:41.756481 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 49595e1db3c44669b438bd0ac1093a62 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d90495f535a64ab390ffd317d36add87]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.756610 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 49595e1db3c44669b438bd0ac1093a62 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d90495f535a64ab390ffd317d36add87]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.756846 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49595e1db3c44669b438bd0ac1093a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.756878 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49595e1db3c44669b438bd0ac1093a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.757313 386 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9aac11ac0b44339b6d9b0f84702871 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d90495f535a64ab390ffd317d36add87]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.757539 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9aac11ac0b44339b6d9b0f84702871. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.759382 2864 tablet_service.cc:1467] Processing CreateTablet for tablet 49595e1db3c44669b438bd0ac1093a62 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d90495f535a64ab390ffd317d36add87]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.759495 4748 tablet_bootstrap.cc:492] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:41.759683 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49595e1db3c44669b438bd0ac1093a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.759981 654 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9aac11ac0b44339b6d9b0f84702871 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d90495f535a64ab390ffd317d36add87]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.760231 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9aac11ac0b44339b6d9b0f84702871. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.760403 4748 tablet_bootstrap.cc:654] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.760962 4747 tablet_bootstrap.cc:492] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:41.762019 4751 tablet_bootstrap.cc:492] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:41.762185 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 66872d5debce41edb013dfcd2c368cb2 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d90495f535a64ab390ffd317d36add87]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.762413 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66872d5debce41edb013dfcd2c368cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.762588 653 tablet_service.cc:1467] Processing CreateTablet for tablet 66872d5debce41edb013dfcd2c368cb2 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d90495f535a64ab390ffd317d36add87]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.762809 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66872d5debce41edb013dfcd2c368cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.762986 4748 tablet_bootstrap.cc:492] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:41.763063 4748 ts_tablet_manager.cc:1367] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:41.763374 4751 tablet_bootstrap.cc:654] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.763563 4748 raft_consensus.cc:348] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.763669 4748 raft_consensus.cc:374] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.763708 4748 raft_consensus.cc:729] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.763856 4748 consensus_queue.cc:260] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.764165 4748 ts_tablet_manager.cc:1398] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:41.761914 4747 tablet_bootstrap.cc:654] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.765702 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 66872d5debce41edb013dfcd2c368cb2 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d90495f535a64ab390ffd317d36add87]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.765522 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9aac11ac0b44339b6d9b0f84702871 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=d90495f535a64ab390ffd317d36add87]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:41.765956 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9aac11ac0b44339b6d9b0f84702871. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.765965 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66872d5debce41edb013dfcd2c368cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:41.765982 4747 tablet_bootstrap.cc:492] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:41.766069 4747 ts_tablet_manager.cc:1367] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:41.766463 4747 raft_consensus.cc:348] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.766567 4747 raft_consensus.cc:374] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.766602 4747 raft_consensus.cc:729] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.766865 4747 consensus_queue.cc:260] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.767217 4747 ts_tablet_manager.cc:1398] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:41.769327 4748 tablet_bootstrap.cc:492] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:41.769714 4749 tablet_bootstrap.cc:492] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:41.771310 4749 tablet_bootstrap.cc:654] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.772588 4747 tablet_bootstrap.cc:492] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:41.773203 4748 tablet_bootstrap.cc:654] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.773270 4751 tablet_bootstrap.cc:492] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:41.773360 4751 ts_tablet_manager.cc:1367] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240502 14:00:41.773497 4752 tablet_bootstrap.cc:492] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:41.773547 4747 tablet_bootstrap.cc:654] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.773835 4751 raft_consensus.cc:348] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.773954 4751 raft_consensus.cc:374] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.773990 4751 raft_consensus.cc:729] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.774101 4751 consensus_queue.cc:260] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:41.774396 4751 ts_tablet_manager.cc:1398] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:41.774616 4751 tablet_bootstrap.cc:492] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:41.775595 4751 tablet_bootstrap.cc:654] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.776329 4752 tablet_bootstrap.cc:654] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.776971 4748 tablet_bootstrap.cc:492] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:41.777099 4748 ts_tablet_manager.cc:1367] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20240502 14:00:41.777415 4752 tablet_bootstrap.cc:492] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:41.777504 4752 ts_tablet_manager.cc:1367] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:41.777724 4751 tablet_bootstrap.cc:492] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:41.777818 4751 ts_tablet_manager.cc:1367] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:41.777962 4752 raft_consensus.cc:348] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:41.778065 4752 raft_consensus.cc:374] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.778106 4752 raft_consensus.cc:729] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.778239 4752 consensus_queue.cc:260] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:41.778295 4751 raft_consensus.cc:348] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.778524 4752 ts_tablet_manager.cc:1398] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:41.778542 4751 raft_consensus.cc:374] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.778587 4751 raft_consensus.cc:729] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.778699 4752 tablet_bootstrap.cc:492] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:41.778698 4751 consensus_queue.cc:260] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.779052 4748 raft_consensus.cc:348] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:41.779151 4748 raft_consensus.cc:374] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.779181 4748 raft_consensus.cc:729] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.779286 4748 consensus_queue.cc:260] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:41.779765 4751 ts_tablet_manager.cc:1398] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:41.779870 4752 tablet_bootstrap.cc:654] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:41.780758 4748 ts_tablet_manager.cc:1398] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:41.781461 4749 tablet_bootstrap.cc:492] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:41.781574 4749 ts_tablet_manager.cc:1367] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 14:00:41.781780 4752 tablet_bootstrap.cc:492] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:41.781888 4752 ts_tablet_manager.cc:1367] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:41.782102 4749 raft_consensus.cc:348] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:41.782220 4749 raft_consensus.cc:374] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.782256 4749 raft_consensus.cc:729] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.782343 4752 raft_consensus.cc:348] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.782433 4752 raft_consensus.cc:374] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.782383 4749 consensus_queue.cc:260] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:41.782464 4752 raft_consensus.cc:729] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.782588 4752 consensus_queue.cc:260] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.782680 4749 ts_tablet_manager.cc:1398] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:41.782842 4752 ts_tablet_manager.cc:1398] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:41.787550 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:41.793650 4744 tablet.cc:1722] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:41.793991 4744 tablet.cc:1620] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:41.794070 4745 tablet.cc:1722] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:41.794132 4746 tablet.cc:1722] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:41.794332 4745 tablet.cc:1620] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:41.794392 4746 tablet.cc:1620] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:41.800479 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=af5660f0891c4705a5ec3823b7b2dc33] alter complete (version 2)
I20240502 14:00:41.804232 4747 tablet_bootstrap.cc:492] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:41.804361 4747 ts_tablet_manager.cc:1367] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20240502 14:00:41.804845 4747 raft_consensus.cc:348] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.804988 4747 raft_consensus.cc:374] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:41.805027 4747 raft_consensus.cc:729] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:41.805156 4747 consensus_queue.cc:260] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:41.805408 4747 ts_tablet_manager.cc:1398] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:41.904727 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:41.909855 4744 tablet.cc:1722] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:41.912451 4746 tablet.cc:1722] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:41.912552 4745 tablet.cc:1722] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:41.917017 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=af5660f0891c4705a5ec3823b7b2dc33] alter complete (version 3)
I20240502 14:00:42.010721 4728 raft_consensus.cc:482] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:42.010831 4728 raft_consensus.cc:504] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.011272 4728 leader_election.cc:288] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:42.011665 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9aac11ac0b44339b6d9b0f84702871" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:42.011684 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9aac11ac0b44339b6d9b0f84702871" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:42.011813 3150 raft_consensus.cc:2454] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:42.011848 405 raft_consensus.cc:2454] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:42.012171 609 leader_election.cc:302] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:42.012426 4728 raft_consensus.cc:2781] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:42.012481 4728 raft_consensus.cc:482] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:42.012527 4728 raft_consensus.cc:3037] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.020880 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:42.021046 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:42.025276 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 80611ba2ee4a495185189c8874c0529b
I20240502 14:00:42.025967 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 80611ba2ee4a495185189c8874c0529b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:42 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:42.026472 4728 raft_consensus.cc:504] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.026875 4728 leader_election.cc:288] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:42.027245 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 80611ba2ee4a495185189c8874c0529b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:42 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:42.027480 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet 80611ba2ee4a495185189c8874c0529b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:42 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:42.027565 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9aac11ac0b44339b6d9b0f84702871" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:42.027688 3150 raft_consensus.cc:3037] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.027864 4764 tablet_replica.cc:330] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:42.028014 4764 raft_consensus.cc:2229] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:42.028337 4764 raft_consensus.cc:2258] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:42.028820 3150 raft_consensus.cc:2454] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:42.028991 4764 ts_tablet_manager.cc:1875] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:42.029532 609 leader_election.cc:302] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:42.030896 4728 raft_consensus.cc:2781] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:42.030993 4728 raft_consensus.cc:686] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:42.031124 4728 consensus_queue.cc:237] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.031589 4764 ts_tablet_manager.cc:1888] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:42.031653 4764 log.cc:1196] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/80611ba2ee4a495185189c8874c0529b
I20240502 14:00:42.031975 4764 ts_tablet_manager.cc:1909] T 80611ba2ee4a495185189c8874c0529b P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:42.032774 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 80611ba2ee4a495185189c8874c0529b (table rebalancer_dynamic_table_new_name [id=af5660f0891c4705a5ec3823b7b2dc33]) successfully deleted
I20240502 14:00:42.033026 4765 tablet_replica.cc:330] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:42.033054 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9aac11ac0b44339b6d9b0f84702871" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:42.033177 4765 raft_consensus.cc:2229] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:42.033195 405 raft_consensus.cc:3037] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.033317 4765 raft_consensus.cc:2258] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:42.033690 32731 catalog_manager.cc:5484] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:42.034282 405 raft_consensus.cc:2454] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:42.034188 4765 ts_tablet_manager.cc:1875] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:42.037674 4765 ts_tablet_manager.cc:1888] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:42.037804 4765 log.cc:1196] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/80611ba2ee4a495185189c8874c0529b
I20240502 14:00:42.038205 4765 ts_tablet_manager.cc:1909] T 80611ba2ee4a495185189c8874c0529b P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:42.038913 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 80611ba2ee4a495185189c8874c0529b (table rebalancer_dynamic_table_new_name [id=af5660f0891c4705a5ec3823b7b2dc33]) successfully deleted
I20240502 14:00:42.043777 4766 tablet_replica.cc:330] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:42.044006 4766 raft_consensus.cc:2229] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:42.044152 4766 raft_consensus.cc:2258] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:42.044754 4766 ts_tablet_manager.cc:1875] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:42.047119 4766 ts_tablet_manager.cc:1888] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:42.047202 4766 log.cc:1196] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/80611ba2ee4a495185189c8874c0529b
I20240502 14:00:42.047590 4766 ts_tablet_manager.cc:1909] T 80611ba2ee4a495185189c8874c0529b P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:42.048688 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 80611ba2ee4a495185189c8874c0529b (table rebalancer_dynamic_table_new_name [id=af5660f0891c4705a5ec3823b7b2dc33]) successfully deleted
I20240502 14:00:42.059182 4194 raft_consensus.cc:482] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:42.059298 4194 raft_consensus.cc:504] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:42.059793 4194 leader_election.cc:288] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:42.060124 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49595e1db3c44669b438bd0ac1093a62" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:42.060220 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49595e1db3c44669b438bd0ac1093a62" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:42.060272 2886 raft_consensus.cc:2454] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:42.060348 3283 raft_consensus.cc:2454] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:42.060570 3087 leader_election.cc:302] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:42.060729 4194 raft_consensus.cc:2781] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:42.060786 4194 raft_consensus.cc:482] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:42.060840 4194 raft_consensus.cc:3037] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.061653 4194 raft_consensus.cc:504] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:42.061954 4194 leader_election.cc:288] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:42.062259 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49595e1db3c44669b438bd0ac1093a62" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:42.062286 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49595e1db3c44669b438bd0ac1093a62" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:42.062355 3283 raft_consensus.cc:3037] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.062389 2886 raft_consensus.cc:3037] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.063136 3283 raft_consensus.cc:2454] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:42.063372 2886 raft_consensus.cc:2454] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:42.063589 3085 leader_election.cc:302] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:42.063740 4194 raft_consensus.cc:2781] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:42.063841 4194 raft_consensus.cc:686] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:42.063987 4194 consensus_queue.cc:237] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:42.065920 32731 catalog_manager.cc:5484] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:42.076643 4370 raft_consensus.cc:482] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:42.076761 4370 raft_consensus.cc:504] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:42.077230 4370 leader_election.cc:288] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:42.077625 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66872d5debce41edb013dfcd2c368cb2" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:42.077662 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66872d5debce41edb013dfcd2c368cb2" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:42.077776 674 raft_consensus.cc:2454] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:42.077813 2886 raft_consensus.cc:2454] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:42.078117 3216 leader_election.cc:302] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:42.078298 4370 raft_consensus.cc:2781] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:42.078370 4370 raft_consensus.cc:482] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:42.078425 4370 raft_consensus.cc:3037] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.079305 4370 raft_consensus.cc:504] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:42.079643 4370 leader_election.cc:288] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:42.080030 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66872d5debce41edb013dfcd2c368cb2" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:42.080081 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66872d5debce41edb013dfcd2c368cb2" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:42.080164 2886 raft_consensus.cc:3037] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.080184 674 raft_consensus.cc:3037] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.081138 2886 raft_consensus.cc:2454] T 66872d5debce41edb013dfcd2c368cb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:42.081138 674 raft_consensus.cc:2454] T 66872d5debce41edb013dfcd2c368cb2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:42.081434 3219 leader_election.cc:302] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:42.081614 4370 raft_consensus.cc:2781] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:42.081701 4370 raft_consensus.cc:686] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:42.081851 4370 consensus_queue.cc:237] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:42.084084 32731 catalog_manager.cc:5484] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:42.129246 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:42.136538 3130 tablet_service.cc:1467] Processing CreateTablet for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=48562df1fe414ade8f8a6a6a079b0185]), partition=
I20240502 14:00:42.136682 653 tablet_service.cc:1467] Processing CreateTablet for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=48562df1fe414ade8f8a6a6a079b0185]), partition=
I20240502 14:00:42.136834 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.136914 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.139091 4752 tablet_bootstrap.cc:492] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:42.139092 4748 tablet_bootstrap.cc:492] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:42.140107 4752 tablet_bootstrap.cc:654] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.140107 4748 tablet_bootstrap.cc:654] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.140914 386 tablet_service.cc:1467] Processing CreateTablet for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=48562df1fe414ade8f8a6a6a079b0185]), partition=
I20240502 14:00:42.141165 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.141547 4748 tablet_bootstrap.cc:492] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:42.141624 4748 ts_tablet_manager.cc:1367] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:42.142055 4748 raft_consensus.cc:348] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:42.142164 4748 raft_consensus.cc:374] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.142194 4748 raft_consensus.cc:729] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.142333 4748 consensus_queue.cc:260] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:42.142644 4748 ts_tablet_manager.cc:1398] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:42.143405 4749 tablet_bootstrap.cc:492] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:42.144315 4749 tablet_bootstrap.cc:654] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.145453 4752 tablet_bootstrap.cc:492] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:42.145547 4752 ts_tablet_manager.cc:1367] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:00:42.146018 4752 raft_consensus.cc:348] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:42.146131 4752 raft_consensus.cc:374] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.146164 4752 raft_consensus.cc:729] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.146283 4752 consensus_queue.cc:260] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:42.146574 4752 ts_tablet_manager.cc:1398] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:42.148792 4749 tablet_bootstrap.cc:492] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:42.148924 4749 ts_tablet_manager.cc:1367] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:42.149426 4749 raft_consensus.cc:348] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:42.149530 4749 raft_consensus.cc:374] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.149564 4749 raft_consensus.cc:729] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.149695 4749 consensus_queue.cc:260] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:42.149987 4749 ts_tablet_manager.cc:1398] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:42.209405 4695 raft_consensus.cc:482] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:42.209543 4695 raft_consensus.cc:504] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:42.210039 4695 leader_election.cc:288] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:42.210362 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ee0edc7a9d4e1f9277c2d5c0aec4d5" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:42.210388 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ee0edc7a9d4e1f9277c2d5c0aec4d5" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:42.210506 674 raft_consensus.cc:2454] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:42.210525 3150 raft_consensus.cc:2454] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:42.210824 344 leader_election.cc:302] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:42.211046 4695 raft_consensus.cc:2781] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:42.211097 4695 raft_consensus.cc:482] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:42.211126 4695 raft_consensus.cc:3037] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.212069 4695 raft_consensus.cc:504] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:42.212392 4695 leader_election.cc:288] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:42.212695 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ee0edc7a9d4e1f9277c2d5c0aec4d5" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:42.212709 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ee0edc7a9d4e1f9277c2d5c0aec4d5" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:42.212801 674 raft_consensus.cc:3037] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.212826 3150 raft_consensus.cc:3037] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.213598 674 raft_consensus.cc:2454] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:42.213886 341 leader_election.cc:302] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:42.213960 3150 raft_consensus.cc:2454] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:42.214094 4695 raft_consensus.cc:2781] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:42.214272 4695 raft_consensus.cc:686] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:42.214422 4695 consensus_queue.cc:237] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:42.216499 32731 catalog_manager.cc:5484] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:42.323415 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:42.327428 3150 raft_consensus.cc:1261] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:42.327450 674 raft_consensus.cc:1261] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:42.327906 4695 consensus_queue.cc:1035] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:42.328081 4773 consensus_queue.cc:1035] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:42.330844 32731 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 (table rebalancer_dynamic_table [id=48562df1fe414ade8f8a6a6a079b0185]). Expected version 1 got 0
I20240502 14:00:42.331701 32732 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 (table rebalancer_dynamic_table [id=48562df1fe414ade8f8a6a6a079b0185]). Expected version 1 got 0
I20240502 14:00:42.332639 4780 tablet.cc:1722] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:42.333030 4780 tablet.cc:1620] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:42.333160 4779 tablet.cc:1722] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:42.333242 32731 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 (table rebalancer_dynamic_table [id=48562df1fe414ade8f8a6a6a079b0185]). Expected version 1 got 0
I20240502 14:00:42.333554 4779 tablet.cc:1620] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:42.334486 32732 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 (table rebalancer_dynamic_table [id=48562df1fe414ade8f8a6a6a079b0185]). Expected version 1 got 0
I20240502 14:00:42.334911 4746 tablet.cc:1722] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:42.335192 4746 tablet.cc:1620] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:42.335690 32732 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 (table rebalancer_dynamic_table [id=48562df1fe414ade8f8a6a6a079b0185]). Expected version 1 got 0
I20240502 14:00:42.339156 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=48562df1fe414ade8f8a6a6a079b0185] alter complete (version 1)
I20240502 14:00:42.442886 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:42.449874 4779 tablet.cc:1722] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:42.450238 4779 tablet.cc:1620] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:42.450289 4746 tablet.cc:1722] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:42.450587 4746 tablet.cc:1620] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:42.456918 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=48562df1fe414ade8f8a6a6a079b0185] alter complete (version 2)
I20240502 14:00:42.456893 4780 tablet.cc:1722] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:42.457973 4780 tablet.cc:1620] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:42.484077 4194 consensus_queue.cc:1035] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:42.487948 4194 consensus_queue.cc:1035] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:42.516863 4728 consensus_queue.cc:1035] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:42.521986 4786 consensus_queue.cc:1035] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:42.558759 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:42.566020 4779 tablet.cc:1722] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:42.569633 4746 tablet.cc:1722] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:42.570258 4780 tablet.cc:1722] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:42.572130 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=48562df1fe414ade8f8a6a6a079b0185] alter complete (version 3)
I20240502 14:00:42.605890 4370 consensus_queue.cc:1035] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:42.611054 4637 consensus_queue.cc:1035] T 66872d5debce41edb013dfcd2c368cb2 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:42.630491 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:42.636850 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa562f43a154941a6e645e04757870d (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d621bb0c4e764e628cf845d1ed77f115]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:42.637221 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa562f43a154941a6e645e04757870d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.637660 2999 tablet_service.cc:1467] Processing CreateTablet for tablet c9d974d536ed41bb9cdc0c3abb64a773 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d621bb0c4e764e628cf845d1ed77f115]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:42.637673 521 tablet_service.cc:1467] Processing CreateTablet for tablet 4ae28ddc2c7346bfb95eaaeae826c0d9 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d621bb0c4e764e628cf845d1ed77f115]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:42.637686 3130 tablet_service.cc:1467] Processing CreateTablet for tablet c9d974d536ed41bb9cdc0c3abb64a773 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d621bb0c4e764e628cf845d1ed77f115]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:42.637921 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ae28ddc2c7346bfb95eaaeae826c0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.637920 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9d974d536ed41bb9cdc0c3abb64a773. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.637964 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9d974d536ed41bb9cdc0c3abb64a773. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.639808 522 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa562f43a154941a6e645e04757870d (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d621bb0c4e764e628cf845d1ed77f115]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:42.640079 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa562f43a154941a6e645e04757870d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.640316 4748 tablet_bootstrap.cc:492] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:42.640291 386 tablet_service.cc:1467] Processing CreateTablet for tablet 4aa562f43a154941a6e645e04757870d (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d621bb0c4e764e628cf845d1ed77f115]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:42.640578 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa562f43a154941a6e645e04757870d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.641275 4748 tablet_bootstrap.cc:654] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.641360 4790 tablet_bootstrap.cc:492] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:42.641948 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 4ae28ddc2c7346bfb95eaaeae826c0d9 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d621bb0c4e764e628cf845d1ed77f115]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:42.642233 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ae28ddc2c7346bfb95eaaeae826c0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.642418 4790 tablet_bootstrap.cc:654] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.642880 4749 tablet_bootstrap.cc:492] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:42.643491 4790 tablet_bootstrap.cc:492] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:42.643569 4790 ts_tablet_manager.cc:1367] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:42.643874 4749 tablet_bootstrap.cc:654] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.644160 4790 raft_consensus.cc:348] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.644277 4790 raft_consensus.cc:374] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.644312 4790 raft_consensus.cc:729] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.644333 4791 tablet_bootstrap.cc:492] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:42.644429 4790 consensus_queue.cc:260] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.644647 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 4ae28ddc2c7346bfb95eaaeae826c0d9 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d621bb0c4e764e628cf845d1ed77f115]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:42.644765 4790 ts_tablet_manager.cc:1398] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:42.644876 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ae28ddc2c7346bfb95eaaeae826c0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.644892 520 tablet_service.cc:1467] Processing CreateTablet for tablet c9d974d536ed41bb9cdc0c3abb64a773 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d621bb0c4e764e628cf845d1ed77f115]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:42.645103 4796 tablet_bootstrap.cc:492] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:42.645172 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9d974d536ed41bb9cdc0c3abb64a773. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.646283 4796 tablet_bootstrap.cc:654] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.647533 4796 tablet_bootstrap.cc:492] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:42.647616 4796 ts_tablet_manager.cc:1367] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:42.648128 4796 raft_consensus.cc:348] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:42.648240 4796 raft_consensus.cc:374] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.648280 4796 raft_consensus.cc:729] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.648406 4796 consensus_queue.cc:260] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:42.648727 4796 ts_tablet_manager.cc:1398] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:42.648741 4794 tablet_bootstrap.cc:492] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:42.649709 4794 tablet_bootstrap.cc:654] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.650806 4794 tablet_bootstrap.cc:492] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:42.650928 4794 ts_tablet_manager.cc:1367] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:42.650967 4791 tablet_bootstrap.cc:654] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.651585 4794 raft_consensus.cc:348] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:42.651715 4794 raft_consensus.cc:374] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.651721 4748 tablet_bootstrap.cc:492] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:42.651775 4794 raft_consensus.cc:729] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.651808 4748 ts_tablet_manager.cc:1367] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 14:00:42.652000 4794 consensus_queue.cc:260] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:42.652315 4748 raft_consensus.cc:348] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:42.652391 4794 ts_tablet_manager.cc:1398] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:42.652428 4748 raft_consensus.cc:374] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.652462 4748 raft_consensus.cc:729] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.652621 4794 tablet_bootstrap.cc:492] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:42.652585 4748 consensus_queue.cc:260] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:42.652859 4791 tablet_bootstrap.cc:492] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:42.652895 4748 ts_tablet_manager.cc:1398] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:42.652936 4791 ts_tablet_manager.cc:1367] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:42.653388 4791 raft_consensus.cc:348] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:42.653492 4791 raft_consensus.cc:374] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.653527 4791 raft_consensus.cc:729] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.653581 4794 tablet_bootstrap.cc:654] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.653631 4791 consensus_queue.cc:260] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:42.653883 4791 ts_tablet_manager.cc:1398] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:42.654042 4791 tablet_bootstrap.cc:492] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:42.655014 4791 tablet_bootstrap.cc:654] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.655942 4749 tablet_bootstrap.cc:492] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:42.656046 4749 ts_tablet_manager.cc:1367] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 14:00:42.656261 4794 tablet_bootstrap.cc:492] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:42.656376 4794 ts_tablet_manager.cc:1367] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:42.656574 4749 raft_consensus.cc:348] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.656697 4749 raft_consensus.cc:374] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.656729 4749 raft_consensus.cc:729] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.656792 4794 raft_consensus.cc:348] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:42.656857 4749 consensus_queue.cc:260] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.656929 4794 raft_consensus.cc:374] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.656973 4794 raft_consensus.cc:729] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.657070 4794 consensus_queue.cc:260] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:42.657167 4749 ts_tablet_manager.cc:1398] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:42.657547 4794 ts_tablet_manager.cc:1398] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:42.657724 4794 tablet_bootstrap.cc:492] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:42.658762 4794 tablet_bootstrap.cc:654] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.660022 4794 tablet_bootstrap.cc:492] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:42.660157 4794 ts_tablet_manager.cc:1367] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:42.660941 4794 raft_consensus.cc:348] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.661110 4794 raft_consensus.cc:374] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.661190 4794 raft_consensus.cc:729] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.661374 4794 consensus_queue.cc:260] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.661639 4794 ts_tablet_manager.cc:1398] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:42.671582 4791 tablet_bootstrap.cc:492] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:42.671694 4791 ts_tablet_manager.cc:1367] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 14:00:42.672192 4791 raft_consensus.cc:348] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:42.672329 4791 raft_consensus.cc:374] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.672361 4791 raft_consensus.cc:729] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.672479 4791 consensus_queue.cc:260] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:42.672742 4791 ts_tablet_manager.cc:1398] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:42.675109 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:42.675268 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:42.677809 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5
I20240502 14:00:42.678319 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:42 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:42.678408 653 tablet_service.cc:1514] Processing DeleteTablet for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:42 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:42.678577 386 tablet_service.cc:1514] Processing DeleteTablet for tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:42 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:42.678784 4803 tablet_replica.cc:330] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:42.678954 4803 raft_consensus.cc:2229] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:42.679091 4803 raft_consensus.cc:2258] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:42.679262 4805 tablet_replica.cc:330] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:42.679302 4804 tablet_replica.cc:330] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:42.679463 4805 raft_consensus.cc:2229] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:42.679704 4805 raft_consensus.cc:2258] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:42.680018 4804 raft_consensus.cc:2229] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:42.680249 4804 raft_consensus.cc:2258] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:42.680439 4805 ts_tablet_manager.cc:1875] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:42.680671 4803 ts_tablet_manager.cc:1875] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:42.682546 4804 ts_tablet_manager.cc:1875] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:42.683120 4803 ts_tablet_manager.cc:1888] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:42.683192 4803 log.cc:1196] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/d5ee0edc7a9d4e1f9277c2d5c0aec4d5
I20240502 14:00:42.683533 4803 ts_tablet_manager.cc:1909] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:42.683667 4805 ts_tablet_manager.cc:1888] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:42.683753 4805 log.cc:1196] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/d5ee0edc7a9d4e1f9277c2d5c0aec4d5
I20240502 14:00:42.684120 4805 ts_tablet_manager.cc:1909] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:42.684229 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 (table rebalancer_dynamic_table_new_name [id=48562df1fe414ade8f8a6a6a079b0185]) successfully deleted
I20240502 14:00:42.684831 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 (table rebalancer_dynamic_table_new_name [id=48562df1fe414ade8f8a6a6a079b0185]) successfully deleted
I20240502 14:00:42.685082 4804 ts_tablet_manager.cc:1888] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:42.685143 4804 log.cc:1196] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/d5ee0edc7a9d4e1f9277c2d5c0aec4d5
I20240502 14:00:42.685374 4804 ts_tablet_manager.cc:1909] T d5ee0edc7a9d4e1f9277c2d5c0aec4d5 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:42.686188 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet d5ee0edc7a9d4e1f9277c2d5c0aec4d5 (table rebalancer_dynamic_table_new_name [id=48562df1fe414ade8f8a6a6a079b0185]) successfully deleted
I20240502 14:00:42.690402 4541 raft_consensus.cc:482] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:42.690497 4541 raft_consensus.cc:504] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:42.690924 4541 leader_election.cc:288] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:42.691267 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ae28ddc2c7346bfb95eaaeae826c0d9" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:42.691407 3018 raft_consensus.cc:2454] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:42.691408 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ae28ddc2c7346bfb95eaaeae826c0d9" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:42.691525 3284 raft_consensus.cc:2454] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:42.691789 475 leader_election.cc:302] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:42.691982 4541 raft_consensus.cc:2781] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:42.692049 4541 raft_consensus.cc:482] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:42.692097 4541 raft_consensus.cc:3037] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.692901 4541 raft_consensus.cc:504] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:42.693269 4541 leader_election.cc:288] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:42.693722 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ae28ddc2c7346bfb95eaaeae826c0d9" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:42.693840 3018 raft_consensus.cc:3037] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.694366 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ae28ddc2c7346bfb95eaaeae826c0d9" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:42.694478 3284 raft_consensus.cc:3037] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.694891 3018 raft_consensus.cc:2454] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:42.695190 3284 raft_consensus.cc:2454] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:42.695461 475 leader_election.cc:302] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:42.695606 4541 raft_consensus.cc:2781] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:42.695688 4541 raft_consensus.cc:686] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:42.695833 4541 consensus_queue.cc:237] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:42.698030 32731 catalog_manager.cc:5484] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:42.757750 4598 raft_consensus.cc:482] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:42.757884 4598 raft_consensus.cc:504] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:42.758375 4598 leader_election.cc:288] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:42.758699 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d974d536ed41bb9cdc0c3abb64a773" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:42.758831 3018 raft_consensus.cc:2454] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:42.758811 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d974d536ed41bb9cdc0c3abb64a773" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:42.758975 541 raft_consensus.cc:2454] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:42.759142 3084 leader_election.cc:302] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:42.759332 4598 raft_consensus.cc:2781] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:42.759392 4598 raft_consensus.cc:482] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:42.759441 4598 raft_consensus.cc:3037] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.760376 4598 raft_consensus.cc:504] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:42.760695 4598 leader_election.cc:288] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:42.760983 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d974d536ed41bb9cdc0c3abb64a773" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:42.761081 3018 raft_consensus.cc:3037] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.761087 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9d974d536ed41bb9cdc0c3abb64a773" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:42.761194 541 raft_consensus.cc:3037] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.761852 3018 raft_consensus.cc:2454] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:42.761932 541 raft_consensus.cc:2454] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:42.762125 3084 leader_election.cc:302] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:42.762266 4598 raft_consensus.cc:2781] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:42.762385 4598 raft_consensus.cc:686] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:42.762540 4598 consensus_queue.cc:237] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:42.764874 32731 catalog_manager.cc:5484] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:42.779911 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:42.781002 4718 raft_consensus.cc:482] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:42.781128 4718 raft_consensus.cc:504] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.781550 4718 leader_election.cc:288] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:42.781922 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa562f43a154941a6e645e04757870d" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:42.782080 541 raft_consensus.cc:2454] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:42.782328 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa562f43a154941a6e645e04757870d" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:42.782469 409 raft_consensus.cc:2454] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:42.782804 2821 leader_election.cc:302] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:42.782963 4718 raft_consensus.cc:2781] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:42.783007 4718 raft_consensus.cc:482] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:42.783059 4718 raft_consensus.cc:3037] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.784103 4718 raft_consensus.cc:504] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.784461 4718 leader_election.cc:288] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:42.784727 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa562f43a154941a6e645e04757870d" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:42.784849 541 raft_consensus.cc:3037] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.785506 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aa562f43a154941a6e645e04757870d" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:42.785630 409 raft_consensus.cc:3037] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.785885 541 raft_consensus.cc:2454] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:42.786226 2819 leader_election.cc:302] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:42.786381 4718 raft_consensus.cc:2781] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:42.786466 4718 raft_consensus.cc:686] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:42.786610 4718 consensus_queue.cc:237] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:42.786651 409 raft_consensus.cc:2454] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:42.788033 386 tablet_service.cc:1467] Processing CreateTablet for tablet 7814c15a76874f9db533c89dfd0d076d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=eb4fa47291094a0f80f3cc15b58f3351]), partition=
I20240502 14:00:42.788038 784 tablet_service.cc:1467] Processing CreateTablet for tablet 7814c15a76874f9db533c89dfd0d076d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=eb4fa47291094a0f80f3cc15b58f3351]), partition=
I20240502 14:00:42.788277 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7814c15a76874f9db533c89dfd0d076d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.788295 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7814c15a76874f9db533c89dfd0d076d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.789979 4749 tablet_bootstrap.cc:492] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:42.790581 522 tablet_service.cc:1467] Processing CreateTablet for tablet 7814c15a76874f9db533c89dfd0d076d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=eb4fa47291094a0f80f3cc15b58f3351]), partition=
I20240502 14:00:42.790827 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7814c15a76874f9db533c89dfd0d076d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:42.790903 4749 tablet_bootstrap.cc:654] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.791678 4809 tablet_bootstrap.cc:492] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:42.792178 4749 tablet_bootstrap.cc:492] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:42.792263 4749 ts_tablet_manager.cc:1367] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:42.792497 32731 catalog_manager.cc:5484] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:42.792713 4749 raft_consensus.cc:348] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:42.792821 4749 raft_consensus.cc:374] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.792852 4749 raft_consensus.cc:729] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.792975 4749 consensus_queue.cc:260] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:42.793277 4749 ts_tablet_manager.cc:1398] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:42.793491 4794 tablet_bootstrap.cc:492] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:42.794384 4794 tablet_bootstrap.cc:654] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.795434 4794 tablet_bootstrap.cc:492] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:42.795522 4794 ts_tablet_manager.cc:1367] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:42.796003 4794 raft_consensus.cc:348] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:42.796103 4794 raft_consensus.cc:374] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.796139 4794 raft_consensus.cc:729] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.796267 4794 consensus_queue.cc:260] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:42.796567 4794 ts_tablet_manager.cc:1398] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:42.798246 4809 tablet_bootstrap.cc:654] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:42.799234 4809 tablet_bootstrap.cc:492] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:42.799325 4809 ts_tablet_manager.cc:1367] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 14:00:42.799706 4809 raft_consensus.cc:348] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:42.799799 4809 raft_consensus.cc:374] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:42.799857 4809 raft_consensus.cc:729] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:42.800024 4809 consensus_queue.cc:260] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:42.800318 4809 ts_tablet_manager.cc:1398] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:42.904659 4541 raft_consensus.cc:482] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:42.904783 4541 raft_consensus.cc:504] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:42.905328 4541 leader_election.cc:288] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:42.905695 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7814c15a76874f9db533c89dfd0d076d" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:42.905756 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7814c15a76874f9db533c89dfd0d076d" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:42.905835 804 raft_consensus.cc:2454] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:42.905915 409 raft_consensus.cc:2454] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:42.906112 477 leader_election.cc:302] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:42.906464 4541 raft_consensus.cc:2781] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:42.906543 4541 raft_consensus.cc:482] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:42.906589 4541 raft_consensus.cc:3037] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.907511 4541 raft_consensus.cc:504] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:42.907903 4541 leader_election.cc:288] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:42.908186 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7814c15a76874f9db533c89dfd0d076d" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:42.908255 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7814c15a76874f9db533c89dfd0d076d" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:42.908310 804 raft_consensus.cc:3037] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.908388 409 raft_consensus.cc:3037] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:42.909138 804 raft_consensus.cc:2454] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:42.909303 409 raft_consensus.cc:2454] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:42.909637 476 leader_election.cc:302] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:42.909817 4541 raft_consensus.cc:2781] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:42.909914 4541 raft_consensus.cc:686] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:42.910517 4541 consensus_queue.cc:237] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:42.912675 32733 catalog_manager.cc:5484] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:43.040663 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:43.044867 409 raft_consensus.cc:1261] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:43.045068 804 raft_consensus.cc:1261] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:43.045356 4541 consensus_queue.cc:1035] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.045524 4762 consensus_queue.cc:1035] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.049715 32733 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet 7814c15a76874f9db533c89dfd0d076d (table rebalancer_dynamic_table [id=eb4fa47291094a0f80f3cc15b58f3351]). Expected version 1 got 0
I20240502 14:00:43.049754 4779 tablet.cc:1722] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:43.048265 4820 tablet.cc:1722] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:43.050097 4779 tablet.cc:1620] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:43.050235 4820 tablet.cc:1620] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:43.051417 32731 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet 7814c15a76874f9db533c89dfd0d076d (table rebalancer_dynamic_table [id=eb4fa47291094a0f80f3cc15b58f3351]). Expected version 1 got 0
I20240502 14:00:43.055300 32731 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 7814c15a76874f9db533c89dfd0d076d (table rebalancer_dynamic_table [id=eb4fa47291094a0f80f3cc15b58f3351]). Expected version 1 got 0
I20240502 14:00:43.055604 32726 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet 7814c15a76874f9db533c89dfd0d076d (table rebalancer_dynamic_table [id=eb4fa47291094a0f80f3cc15b58f3351]). Expected version 1 got 0
I20240502 14:00:43.056887 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=eb4fa47291094a0f80f3cc15b58f3351] alter complete (version 1)
I20240502 14:00:43.059669 4821 tablet.cc:1722] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:43.059988 4821 tablet.cc:1620] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:43.076730 4762 consensus_queue.cc:1035] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.081053 4541 consensus_queue.cc:1035] T 4ae28ddc2c7346bfb95eaaeae826c0d9 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.159780 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:43.164896 4820 tablet.cc:1722] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:43.165271 4820 tablet.cc:1620] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:43.165786 4779 tablet.cc:1722] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:43.166096 4779 tablet.cc:1620] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:43.168277 4821 tablet.cc:1722] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:43.168578 4821 tablet.cc:1620] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:43.171149 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=eb4fa47291094a0f80f3cc15b58f3351] alter complete (version 2)
I20240502 14:00:43.186669 4815 consensus_queue.cc:1035] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.189854 4718 consensus_queue.cc:1035] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.237401 4814 consensus_queue.cc:1035] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.241598 4194 consensus_queue.cc:1035] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.274845 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:43.280380 4820 tablet.cc:1722] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:43.281025 4821 tablet.cc:1722] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:43.283278 4779 tablet.cc:1722] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:43.287103 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=eb4fa47291094a0f80f3cc15b58f3351] alter complete (version 3)
I20240502 14:00:43.308874 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:43.314821 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 093babce464c4152ab6ade948dc7a616 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=621920b4c989408eb3ab52ac1405f687]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:43.314878 522 tablet_service.cc:1467] Processing CreateTablet for tablet 093babce464c4152ab6ade948dc7a616 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=621920b4c989408eb3ab52ac1405f687]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:43.315174 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 093babce464c4152ab6ade948dc7a616. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.315193 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 093babce464c4152ab6ade948dc7a616. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.315562 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 093babce464c4152ab6ade948dc7a616 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=621920b4c989408eb3ab52ac1405f687]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:43.315810 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 093babce464c4152ab6ade948dc7a616. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.317157 4834 tablet_bootstrap.cc:492] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:43.317314 520 tablet_service.cc:1467] Processing CreateTablet for tablet 7450ad74adcc49c3820f93b568ee7c21 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=621920b4c989408eb3ab52ac1405f687]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:43.317533 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7450ad74adcc49c3820f93b568ee7c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.318255 4837 tablet_bootstrap.cc:492] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:43.318549 4835 tablet_bootstrap.cc:492] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:43.319317 4837 tablet_bootstrap.cc:654] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.319864 653 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcc29071c964817ac2b517c563027ab (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=621920b4c989408eb3ab52ac1405f687]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:43.320045 4834 tablet_bootstrap.cc:654] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.320165 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcc29071c964817ac2b517c563027ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.321527 4835 tablet_bootstrap.cc:654] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.321578 4834 tablet_bootstrap.cc:492] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:43.321718 4834 ts_tablet_manager.cc:1367] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:43.322244 4834 raft_consensus.cc:348] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:43.322537 4840 tablet_bootstrap.cc:492] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:43.323017 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 7450ad74adcc49c3820f93b568ee7c21 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=621920b4c989408eb3ab52ac1405f687]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:43.323187 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcc29071c964817ac2b517c563027ab (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=621920b4c989408eb3ab52ac1405f687]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:43.323283 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7450ad74adcc49c3820f93b568ee7c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.323376 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcc29071c964817ac2b517c563027ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.323606 4840 tablet_bootstrap.cc:654] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.324815 4837 tablet_bootstrap.cc:492] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:43.324880 4837 ts_tablet_manager.cc:1367] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:43.325225 4837 raft_consensus.cc:348] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:43.325335 4837 raft_consensus.cc:374] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.325377 4837 raft_consensus.cc:729] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.325520 4837 consensus_queue.cc:260] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:43.325585 4843 tablet_bootstrap.cc:492] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:43.325608 784 tablet_service.cc:1467] Processing CreateTablet for tablet 7450ad74adcc49c3820f93b568ee7c21 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=621920b4c989408eb3ab52ac1405f687]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:43.325805 4837 ts_tablet_manager.cc:1398] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:43.325893 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7450ad74adcc49c3820f93b568ee7c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.326465 4843 tablet_bootstrap.cc:654] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.326471 4834 raft_consensus.cc:374] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.326838 4834 raft_consensus.cc:729] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.327090 4834 consensus_queue.cc:260] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:43.327422 4834 ts_tablet_manager.cc:1398] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:00:43.328480 4843 tablet_bootstrap.cc:492] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:43.328620 4843 ts_tablet_manager.cc:1367] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:43.328894 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcc29071c964817ac2b517c563027ab (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=621920b4c989408eb3ab52ac1405f687]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:43.329139 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcc29071c964817ac2b517c563027ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.329109 4843 raft_consensus.cc:348] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.329206 4843 raft_consensus.cc:374] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.329236 4843 raft_consensus.cc:729] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.329370 4843 consensus_queue.cc:260] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.329674 4843 ts_tablet_manager.cc:1398] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:43.330039 4840 tablet_bootstrap.cc:492] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:43.331285 4843 tablet_bootstrap.cc:492] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:43.330178 4840 ts_tablet_manager.cc:1367] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:43.332316 4834 tablet_bootstrap.cc:492] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:43.332434 4840 raft_consensus.cc:348] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.332551 4840 raft_consensus.cc:374] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.332588 4840 raft_consensus.cc:729] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.332710 4840 consensus_queue.cc:260] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.332998 4840 ts_tablet_manager.cc:1398] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:43.333205 4843 tablet_bootstrap.cc:654] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.333205 4845 tablet_bootstrap.cc:492] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:43.333253 4834 tablet_bootstrap.cc:654] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.334178 4845 tablet_bootstrap.cc:654] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.334313 4843 tablet_bootstrap.cc:492] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:43.334424 4843 ts_tablet_manager.cc:1367] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:43.335140 4843 raft_consensus.cc:348] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:43.335331 4843 raft_consensus.cc:374] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.335419 4843 raft_consensus.cc:729] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.335656 4845 tablet_bootstrap.cc:492] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:43.335747 4845 ts_tablet_manager.cc:1367] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:43.335794 4843 consensus_queue.cc:260] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:43.336237 4845 raft_consensus.cc:348] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:43.336349 4845 raft_consensus.cc:374] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.336380 4845 raft_consensus.cc:729] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.336540 4845 consensus_queue.cc:260] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:43.336886 4845 ts_tablet_manager.cc:1398] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:43.336896 4843 ts_tablet_manager.cc:1398] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:43.337776 4835 tablet_bootstrap.cc:492] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:43.337875 4835 ts_tablet_manager.cc:1367] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 14:00:43.338377 4835 raft_consensus.cc:348] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:43.338466 4835 raft_consensus.cc:374] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.338487 4835 raft_consensus.cc:729] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.338604 4835 consensus_queue.cc:260] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:43.338811 4834 tablet_bootstrap.cc:492] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:43.338864 4835 ts_tablet_manager.cc:1398] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:43.338900 4834 ts_tablet_manager.cc:1367] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:43.339066 4835 tablet_bootstrap.cc:492] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:43.339401 4834 raft_consensus.cc:348] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.339524 4834 raft_consensus.cc:374] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.339560 4834 raft_consensus.cc:729] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.339679 4834 consensus_queue.cc:260] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.339982 4834 ts_tablet_manager.cc:1398] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:43.340047 4835 tablet_bootstrap.cc:654] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.341814 4835 tablet_bootstrap.cc:492] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:43.341939 4835 ts_tablet_manager.cc:1367] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:43.342567 4835 raft_consensus.cc:348] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:43.342739 4835 raft_consensus.cc:374] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.342820 4835 raft_consensus.cc:729] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.343005 4835 consensus_queue.cc:260] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:43.343550 4835 ts_tablet_manager.cc:1398] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:43.352357 4815 raft_consensus.cc:482] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:43.352466 4815 raft_consensus.cc:504] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.352905 4815 leader_election.cc:288] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:43.353212 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcc29071c964817ac2b517c563027ab" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:43.353355 3151 raft_consensus.cc:2454] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:43.353782 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcc29071c964817ac2b517c563027ab" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:43.353936 674 raft_consensus.cc:2454] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:43.354177 2822 leader_election.cc:302] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:43.354405 4815 raft_consensus.cc:2781] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:43.354481 4815 raft_consensus.cc:482] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:43.354530 4815 raft_consensus.cc:3037] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.355405 4815 raft_consensus.cc:504] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.355779 4815 leader_election.cc:288] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:43.356091 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcc29071c964817ac2b517c563027ab" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:43.356233 3151 raft_consensus.cc:3037] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.356302 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcc29071c964817ac2b517c563027ab" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:43.356405 674 raft_consensus.cc:3037] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.357174 674 raft_consensus.cc:2454] T 2fcc29071c964817ac2b517c563027ab P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:43.357189 3151 raft_consensus.cc:2454] T 2fcc29071c964817ac2b517c563027ab P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:43.357486 2819 leader_election.cc:302] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:43.357671 4815 raft_consensus.cc:2781] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:43.357755 4815 raft_consensus.cc:686] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:43.357892 4815 consensus_queue.cc:237] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.360033 32732 catalog_manager.cc:5484] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.361222 4833 raft_consensus.cc:482] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:43.361326 4833 raft_consensus.cc:504] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:43.361712 4833 leader_election.cc:288] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:43.361984 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093babce464c4152ab6ade948dc7a616" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:43.362002 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093babce464c4152ab6ade948dc7a616" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:43.362092 3019 raft_consensus.cc:2454] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:43.362123 2886 raft_consensus.cc:2454] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:43.362366 474 leader_election.cc:302] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:43.362560 4833 raft_consensus.cc:2781] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:43.362610 4833 raft_consensus.cc:482] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:43.362655 4833 raft_consensus.cc:3037] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.363461 4833 raft_consensus.cc:504] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:43.363775 4833 leader_election.cc:288] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:43.364104 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093babce464c4152ab6ade948dc7a616" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:43.364202 2886 raft_consensus.cc:3037] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.364226 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093babce464c4152ab6ade948dc7a616" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:43.364351 3019 raft_consensus.cc:3037] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.365151 2886 raft_consensus.cc:2454] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:43.365211 3019 raft_consensus.cc:2454] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:43.365504 477 leader_election.cc:302] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:43.365674 4833 raft_consensus.cc:2781] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:43.365760 4833 raft_consensus.cc:686] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:43.365897 4833 consensus_queue.cc:237] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:43.368317 32732 catalog_manager.cc:5484] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.388056 32732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:43.388216 32732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:43.390307 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 7814c15a76874f9db533c89dfd0d076d
I20240502 14:00:43.390880 386 tablet_service.cc:1514] Processing DeleteTablet for tablet 7814c15a76874f9db533c89dfd0d076d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:43 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:43.390887 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 7814c15a76874f9db533c89dfd0d076d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:43 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:43.391145 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 7814c15a76874f9db533c89dfd0d076d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:43 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:43.391242 4852 tablet_replica.cc:330] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:43.391417 4852 raft_consensus.cc:2229] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:43.391541 4854 tablet_replica.cc:330] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:43.391625 4852 raft_consensus.cc:2258] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:43.391742 4854 raft_consensus.cc:2229] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:43.391880 4853 tablet_replica.cc:330] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:43.392107 4853 raft_consensus.cc:2229] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:43.392247 4854 raft_consensus.cc:2258] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:43.392287 4853 raft_consensus.cc:2258] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:43.392434 4852 ts_tablet_manager.cc:1875] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:43.392896 4854 ts_tablet_manager.cc:1875] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:43.392930 4853 ts_tablet_manager.cc:1875] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:43.394881 4852 ts_tablet_manager.cc:1888] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:43.395025 4852 log.cc:1196] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/7814c15a76874f9db533c89dfd0d076d
I20240502 14:00:43.395257 4854 ts_tablet_manager.cc:1888] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:43.395404 4854 log.cc:1196] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/7814c15a76874f9db533c89dfd0d076d
I20240502 14:00:43.395445 4852 ts_tablet_manager.cc:1909] T 7814c15a76874f9db533c89dfd0d076d P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:43.395684 4853 ts_tablet_manager.cc:1888] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:43.395802 4853 log.cc:1196] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/7814c15a76874f9db533c89dfd0d076d
I20240502 14:00:43.395787 4854 ts_tablet_manager.cc:1909] T 7814c15a76874f9db533c89dfd0d076d P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:43.396185 4853 ts_tablet_manager.cc:1909] T 7814c15a76874f9db533c89dfd0d076d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:43.396252 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 7814c15a76874f9db533c89dfd0d076d (table rebalancer_dynamic_table_new_name [id=eb4fa47291094a0f80f3cc15b58f3351]) successfully deleted
I20240502 14:00:43.396714 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 7814c15a76874f9db533c89dfd0d076d (table rebalancer_dynamic_table_new_name [id=eb4fa47291094a0f80f3cc15b58f3351]) successfully deleted
I20240502 14:00:43.397274 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 7814c15a76874f9db533c89dfd0d076d (table rebalancer_dynamic_table_new_name [id=eb4fa47291094a0f80f3cc15b58f3351]) successfully deleted
I20240502 14:00:43.492622 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:43.498239 3263 tablet_service.cc:1467] Processing CreateTablet for tablet ac1c3354a828480f9f73090afe1be0af (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2f8362c5381c494783bf9cd005c28191]), partition=
I20240502 14:00:43.498426 386 tablet_service.cc:1467] Processing CreateTablet for tablet ac1c3354a828480f9f73090afe1be0af (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2f8362c5381c494783bf9cd005c28191]), partition=
I20240502 14:00:43.498504 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac1c3354a828480f9f73090afe1be0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.498687 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac1c3354a828480f9f73090afe1be0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.500105 653 tablet_service.cc:1467] Processing CreateTablet for tablet ac1c3354a828480f9f73090afe1be0af (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2f8362c5381c494783bf9cd005c28191]), partition=
I20240502 14:00:43.500351 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac1c3354a828480f9f73090afe1be0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.500687 4855 tablet_bootstrap.cc:492] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:43.500972 4856 tablet_bootstrap.cc:492] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:43.501448 4855 tablet_bootstrap.cc:654] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.501915 4856 tablet_bootstrap.cc:654] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.502554 4840 tablet_bootstrap.cc:492] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:43.503430 4840 tablet_bootstrap.cc:654] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:43.503805 4855 tablet_bootstrap.cc:492] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:43.503906 4855 ts_tablet_manager.cc:1367] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:43.504490 4855 raft_consensus.cc:348] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.504623 4855 raft_consensus.cc:374] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.504658 4855 raft_consensus.cc:729] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.504809 4855 consensus_queue.cc:260] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.505018 4840 tablet_bootstrap.cc:492] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:43.505097 4840 ts_tablet_manager.cc:1367] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:43.505484 4840 raft_consensus.cc:348] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.505576 4840 raft_consensus.cc:374] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.505604 4840 raft_consensus.cc:729] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.505702 4840 consensus_queue.cc:260] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.505970 4840 ts_tablet_manager.cc:1398] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:43.505851 4855 ts_tablet_manager.cc:1398] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:43.507606 4856 tablet_bootstrap.cc:492] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:43.507697 4856 ts_tablet_manager.cc:1367] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:43.508229 4856 raft_consensus.cc:348] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.508354 4856 raft_consensus.cc:374] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:43.508394 4856 raft_consensus.cc:729] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:43.508524 4856 consensus_queue.cc:260] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.508823 4856 ts_tablet_manager.cc:1398] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:43.627766 4598 raft_consensus.cc:482] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:43.627920 4598 raft_consensus.cc:504] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:43.628369 4598 leader_election.cc:288] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:43.628710 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7450ad74adcc49c3820f93b568ee7c21" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:43.628846 804 raft_consensus.cc:2454] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:43.628852 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7450ad74adcc49c3820f93b568ee7c21" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:43.629029 541 raft_consensus.cc:2454] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:43.629142 3087 leader_election.cc:302] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:43.629325 4598 raft_consensus.cc:2781] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:43.629387 4598 raft_consensus.cc:482] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:43.629444 4598 raft_consensus.cc:3037] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.630239 4598 raft_consensus.cc:504] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:43.630519 4598 leader_election.cc:288] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:43.630838 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7450ad74adcc49c3820f93b568ee7c21" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:43.630865 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7450ad74adcc49c3820f93b568ee7c21" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:43.630934 804 raft_consensus.cc:3037] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.630980 541 raft_consensus.cc:3037] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.631716 804 raft_consensus.cc:2454] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:43.631903 541 raft_consensus.cc:2454] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:43.632093 3087 leader_election.cc:302] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:43.632272 4598 raft_consensus.cc:2781] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:43.632442 4598 raft_consensus.cc:686] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:43.632580 4598 consensus_queue.cc:237] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:43.634689 32726 catalog_manager.cc:5484] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.652926 4773 raft_consensus.cc:482] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:43.653054 4773 raft_consensus.cc:504] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.653438 4773 leader_election.cc:288] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:43.653757 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac1c3354a828480f9f73090afe1be0af" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:43.653795 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac1c3354a828480f9f73090afe1be0af" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:43.653908 674 raft_consensus.cc:2454] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:43.653908 3284 raft_consensus.cc:2454] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:43.654166 341 leader_election.cc:302] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:43.654327 4773 raft_consensus.cc:2781] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:43.654392 4773 raft_consensus.cc:482] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:43.654445 4773 raft_consensus.cc:3037] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.655408 4773 raft_consensus.cc:504] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.655783 4773 leader_election.cc:288] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:43.656153 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac1c3354a828480f9f73090afe1be0af" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:43.656186 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac1c3354a828480f9f73090afe1be0af" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:43.656273 3284 raft_consensus.cc:3037] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.656306 674 raft_consensus.cc:3037] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:43.657243 3284 raft_consensus.cc:2454] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:43.657243 674 raft_consensus.cc:2454] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:43.659060 341 leader_election.cc:302] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:43.659240 4773 raft_consensus.cc:2781] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:43.659329 4773 raft_consensus.cc:686] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:43.659488 4773 consensus_queue.cc:237] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:43.662001 32726 catalog_manager.cc:5484] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.746769 4833 consensus_queue.cc:1035] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.751093 4833 consensus_queue.cc:1035] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.784914 4657 consensus_queue.cc:1035] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.787776 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:43.790012 4815 consensus_queue.cc:1035] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.793169 3284 raft_consensus.cc:1261] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:43.793908 4773 consensus_queue.cc:1035] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.794369 672 raft_consensus.cc:1261] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:43.795631 4773 consensus_queue.cc:1035] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:43.798467 32731 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet ac1c3354a828480f9f73090afe1be0af (table rebalancer_dynamic_table [id=2f8362c5381c494783bf9cd005c28191]). Expected version 1 got 0
I20240502 14:00:43.798825 32726 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet ac1c3354a828480f9f73090afe1be0af (table rebalancer_dynamic_table [id=2f8362c5381c494783bf9cd005c28191]). Expected version 1 got 0
I20240502 14:00:43.799458 4875 tablet.cc:1722] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:43.799774 4875 tablet.cc:1620] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:43.800721 4876 tablet.cc:1722] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:43.801801 32726 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet ac1c3354a828480f9f73090afe1be0af (table rebalancer_dynamic_table [id=2f8362c5381c494783bf9cd005c28191]). Expected version 1 got 0
I20240502 14:00:43.802047 4876 tablet.cc:1620] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:43.802151 32731 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet ac1c3354a828480f9f73090afe1be0af (table rebalancer_dynamic_table [id=2f8362c5381c494783bf9cd005c28191]). Expected version 1 got 0
I20240502 14:00:43.803098 32732 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet ac1c3354a828480f9f73090afe1be0af (table rebalancer_dynamic_table [id=2f8362c5381c494783bf9cd005c28191]). Expected version 1 got 0
I20240502 14:00:43.804881 4878 tablet.cc:1722] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:43.805444 4878 tablet.cc:1620] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:43.806267 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=2f8362c5381c494783bf9cd005c28191] alter complete (version 1)
I20240502 14:00:43.907722 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:43.913764 4875 tablet.cc:1722] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:43.914168 4875 tablet.cc:1620] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:43.914273 4878 tablet.cc:1722] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:43.914681 4878 tablet.cc:1620] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:43.915117 4876 tablet.cc:1722] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:43.915395 4876 tablet.cc:1620] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:43.920012 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=2f8362c5381c494783bf9cd005c28191] alter complete (version 2)
I20240502 14:00:44.023726 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:44.030391 4194 consensus_queue.cc:1035] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:44.033597 4875 tablet.cc:1722] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:44.034036 4878 tablet.cc:1722] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:44.039376 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=2f8362c5381c494783bf9cd005c28191] alter complete (version 3)
I20240502 14:00:44.041507 4876 tablet.cc:1722] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:44.045959 4814 consensus_queue.cc:1035] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:44.141741 32726 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:44.141911 32726 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:44.144304 32726 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet ac1c3354a828480f9f73090afe1be0af
I20240502 14:00:44.144850 386 tablet_service.cc:1514] Processing DeleteTablet for tablet ac1c3354a828480f9f73090afe1be0af with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:44 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:44.145020 653 tablet_service.cc:1514] Processing DeleteTablet for tablet ac1c3354a828480f9f73090afe1be0af with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:44 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:44.145166 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet ac1c3354a828480f9f73090afe1be0af with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:44 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:44.145315 4884 tablet_replica.cc:330] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:44.145470 4885 tablet_replica.cc:330] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:44.145496 4884 raft_consensus.cc:2229] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:44.145545 4883 tablet_replica.cc:330] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:44.145645 4884 raft_consensus.cc:2258] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:44.145689 4885 raft_consensus.cc:2229] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:44.145716 4883 raft_consensus.cc:2229] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:44.146000 4883 raft_consensus.cc:2258] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:44.146059 4885 raft_consensus.cc:2258] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:44.146611 4884 ts_tablet_manager.cc:1875] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:44.146620 4883 ts_tablet_manager.cc:1875] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:44.146669 4885 ts_tablet_manager.cc:1875] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:44.149034 4884 ts_tablet_manager.cc:1888] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:44.149089 4883 ts_tablet_manager.cc:1888] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:44.149147 4884 log.cc:1196] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/ac1c3354a828480f9f73090afe1be0af
I20240502 14:00:44.149192 4883 log.cc:1196] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/ac1c3354a828480f9f73090afe1be0af
I20240502 14:00:44.149470 4885 ts_tablet_manager.cc:1888] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:44.149516 4884 ts_tablet_manager.cc:1909] T ac1c3354a828480f9f73090afe1be0af P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:44.149535 4883 ts_tablet_manager.cc:1909] T ac1c3354a828480f9f73090afe1be0af P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:44.149585 4885 log.cc:1196] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/ac1c3354a828480f9f73090afe1be0af
I20240502 14:00:44.149952 4885 ts_tablet_manager.cc:1909] T ac1c3354a828480f9f73090afe1be0af P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:44.150246 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet ac1c3354a828480f9f73090afe1be0af (table rebalancer_dynamic_table_new_name [id=2f8362c5381c494783bf9cd005c28191]) successfully deleted
I20240502 14:00:44.150337 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet ac1c3354a828480f9f73090afe1be0af (table rebalancer_dynamic_table_new_name [id=2f8362c5381c494783bf9cd005c28191]) successfully deleted
I20240502 14:00:44.150660 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet ac1c3354a828480f9f73090afe1be0af (table rebalancer_dynamic_table_new_name [id=2f8362c5381c494783bf9cd005c28191]) successfully deleted
I20240502 14:00:44.197225 32726 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:44.204032 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 9e642731d7684e33b82eabf97c380123 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=c36d55efbf0d492381cd64ae4e1f17e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:44.204205 784 tablet_service.cc:1467] Processing CreateTablet for tablet 9e642731d7684e33b82eabf97c380123 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=c36d55efbf0d492381cd64ae4e1f17e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:44.204341 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 9e642731d7684e33b82eabf97c380123 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=c36d55efbf0d492381cd64ae4e1f17e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:44.204386 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e642731d7684e33b82eabf97c380123. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.204525 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e642731d7684e33b82eabf97c380123. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.204602 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e642731d7684e33b82eabf97c380123. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.204687 386 tablet_service.cc:1467] Processing CreateTablet for tablet 6115d4bcf9e04e7b84e251c8facb2bdf (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=c36d55efbf0d492381cd64ae4e1f17e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:44.204980 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6115d4bcf9e04e7b84e251c8facb2bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.207010 388 tablet_service.cc:1467] Processing CreateTablet for tablet 8c73305a96554a31a68e60e2207232fe (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=c36d55efbf0d492381cd64ae4e1f17e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:44.207063 520 tablet_service.cc:1467] Processing CreateTablet for tablet 6115d4bcf9e04e7b84e251c8facb2bdf (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=c36d55efbf0d492381cd64ae4e1f17e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:44.207254 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c73305a96554a31a68e60e2207232fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.207341 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6115d4bcf9e04e7b84e251c8facb2bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.207404 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 6115d4bcf9e04e7b84e251c8facb2bdf (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=c36d55efbf0d492381cd64ae4e1f17e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:44.207679 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6115d4bcf9e04e7b84e251c8facb2bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.207744 4888 tablet_bootstrap.cc:492] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:44.210592 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 8c73305a96554a31a68e60e2207232fe (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=c36d55efbf0d492381cd64ae4e1f17e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:44.210866 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c73305a96554a31a68e60e2207232fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.211131 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 8c73305a96554a31a68e60e2207232fe (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=c36d55efbf0d492381cd64ae4e1f17e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:44.211392 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c73305a96554a31a68e60e2207232fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.211434 4888 tablet_bootstrap.cc:654] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.212956 4891 tablet_bootstrap.cc:492] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:44.213670 4886 tablet_bootstrap.cc:492] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:44.214883 4886 tablet_bootstrap.cc:654] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.214934 4893 tablet_bootstrap.cc:492] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:44.215101 4890 tablet_bootstrap.cc:492] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:44.215951 4893 tablet_bootstrap.cc:654] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.215957 4887 tablet_bootstrap.cc:492] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:44.216142 4890 tablet_bootstrap.cc:654] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.216202 4889 tablet_bootstrap.cc:492] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:44.216306 4891 tablet_bootstrap.cc:654] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.217226 4887 tablet_bootstrap.cc:654] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.217890 4891 tablet_bootstrap.cc:492] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:44.217993 4891 ts_tablet_manager.cc:1367] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:44.218528 4891 raft_consensus.cc:348] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.218650 4891 raft_consensus.cc:374] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.218684 4891 raft_consensus.cc:729] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.218748 4886 tablet_bootstrap.cc:492] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:44.218811 4891 consensus_queue.cc:260] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.218863 4886 ts_tablet_manager.cc:1367] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:44.218999 4893 tablet_bootstrap.cc:492] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:44.219120 4891 ts_tablet_manager.cc:1398] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:44.219400 4886 raft_consensus.cc:348] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:44.219530 4886 raft_consensus.cc:374] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.219570 4886 raft_consensus.cc:729] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.219700 4893 ts_tablet_manager.cc:1367] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 14:00:44.219708 4886 consensus_queue.cc:260] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:44.220136 4890 tablet_bootstrap.cc:492] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:44.220278 4890 ts_tablet_manager.cc:1367] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:44.220535 4888 tablet_bootstrap.cc:492] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:44.220640 4888 ts_tablet_manager.cc:1367] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 14:00:44.220778 4890 raft_consensus.cc:348] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.220888 4890 raft_consensus.cc:374] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.220925 4890 raft_consensus.cc:729] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.221055 4890 consensus_queue.cc:260] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.221149 4886 ts_tablet_manager.cc:1398] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:44.221148 4888 raft_consensus.cc:348] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:44.221261 4888 raft_consensus.cc:374] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.221293 4888 raft_consensus.cc:729] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.221413 4888 consensus_queue.cc:260] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:44.221707 4888 ts_tablet_manager.cc:1398] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:44.221864 4887 tablet_bootstrap.cc:492] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:44.221963 4887 ts_tablet_manager.cc:1367] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:44.222391 4890 ts_tablet_manager.cc:1398] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:44.222434 4887 raft_consensus.cc:348] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:44.222541 4887 raft_consensus.cc:374] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.222571 4887 raft_consensus.cc:729] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.222693 4887 consensus_queue.cc:260] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:44.222997 4887 ts_tablet_manager.cc:1398] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:44.223155 4887 tablet_bootstrap.cc:492] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:44.224119 4887 tablet_bootstrap.cc:654] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.220890 4889 tablet_bootstrap.cc:654] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.226163 4893 raft_consensus.cc:348] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:44.227380 4893 raft_consensus.cc:374] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.227988 4893 raft_consensus.cc:729] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.228313 4893 consensus_queue.cc:260] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:44.228942 4893 ts_tablet_manager.cc:1398] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.009s user 0.000s sys 0.001s
I20240502 14:00:44.230008 4887 tablet_bootstrap.cc:492] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:44.230162 4887 ts_tablet_manager.cc:1367] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:44.230686 4887 raft_consensus.cc:348] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:44.230810 4887 raft_consensus.cc:374] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.230849 4887 raft_consensus.cc:729] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.230979 4887 consensus_queue.cc:260] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:44.231307 4887 ts_tablet_manager.cc:1398] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:44.232189 4889 tablet_bootstrap.cc:492] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:44.232291 4889 ts_tablet_manager.cc:1367] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 14:00:44.232774 4889 raft_consensus.cc:348] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.232884 4889 raft_consensus.cc:374] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.232925 4889 raft_consensus.cc:729] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.233045 4889 consensus_queue.cc:260] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.233361 4889 ts_tablet_manager.cc:1398] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:44.233865 4889 tablet_bootstrap.cc:492] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:44.235244 4889 tablet_bootstrap.cc:654] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.236483 4889 tablet_bootstrap.cc:492] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:44.236562 4889 ts_tablet_manager.cc:1367] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:44.237066 4889 raft_consensus.cc:348] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:44.237170 4889 raft_consensus.cc:374] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.237206 4889 raft_consensus.cc:729] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.237332 4889 consensus_queue.cc:260] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:44.237597 4889 ts_tablet_manager.cc:1398] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:44.245844 4814 raft_consensus.cc:482] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:44.245980 4814 raft_consensus.cc:504] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.246539 4814 leader_election.cc:288] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:44.247045 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6115d4bcf9e04e7b84e251c8facb2bdf" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:44.247186 409 raft_consensus.cc:2454] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:44.247113 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:44.247969 3086 leader_election.cc:302] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:44.248494 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6115d4bcf9e04e7b84e251c8facb2bdf" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:44.248518 4598 raft_consensus.cc:2781] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:44.248597 4598 raft_consensus.cc:482] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:44.248661 4598 raft_consensus.cc:3037] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.248661 541 raft_consensus.cc:2454] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:44.249846 4598 raft_consensus.cc:504] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.250223 4598 leader_election.cc:288] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:44.250674 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6115d4bcf9e04e7b84e251c8facb2bdf" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:44.250793 409 raft_consensus.cc:3037] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.250849 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6115d4bcf9e04e7b84e251c8facb2bdf" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:44.250963 541 raft_consensus.cc:3037] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.251569 409 raft_consensus.cc:2454] T 6115d4bcf9e04e7b84e251c8facb2bdf P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:44.251901 3086 leader_election.cc:302] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:44.251920 541 raft_consensus.cc:2454] T 6115d4bcf9e04e7b84e251c8facb2bdf P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:44.252233 4814 raft_consensus.cc:2781] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:44.252334 4814 raft_consensus.cc:686] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:44.252485 4814 consensus_queue.cc:237] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.257504 653 tablet_service.cc:1467] Processing CreateTablet for tablet 04393d9da74844acbbb13a17c52905b7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c5040325e91744e28742be8db85f46dd]), partition=
I20240502 14:00:44.257782 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04393d9da74844acbbb13a17c52905b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.257766 520 tablet_service.cc:1467] Processing CreateTablet for tablet 04393d9da74844acbbb13a17c52905b7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c5040325e91744e28742be8db85f46dd]), partition=
I20240502 14:00:44.257979 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04393d9da74844acbbb13a17c52905b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.258213 32731 catalog_manager.cc:5484] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:44.260296 784 tablet_service.cc:1467] Processing CreateTablet for tablet 04393d9da74844acbbb13a17c52905b7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c5040325e91744e28742be8db85f46dd]), partition=
I20240502 14:00:44.260531 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04393d9da74844acbbb13a17c52905b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.262387 4886 tablet_bootstrap.cc:492] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:44.263334 4886 tablet_bootstrap.cc:654] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.263348 4890 tablet_bootstrap.cc:492] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:44.264361 4890 tablet_bootstrap.cc:654] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.265528 4890 tablet_bootstrap.cc:492] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:44.265621 4890 ts_tablet_manager.cc:1367] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:44.265775 4903 tablet_bootstrap.cc:492] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:44.266742 4903 tablet_bootstrap.cc:654] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.268000 4903 tablet_bootstrap.cc:492] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:44.268080 4903 ts_tablet_manager.cc:1367] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:44.268424 4890 raft_consensus.cc:348] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.268540 4890 raft_consensus.cc:374] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.268573 4890 raft_consensus.cc:729] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.268503 4903 raft_consensus.cc:348] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.268620 4903 raft_consensus.cc:374] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.268654 4903 raft_consensus.cc:729] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.268702 4890 consensus_queue.cc:260] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.268769 4903 consensus_queue.cc:260] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.269155 4890 ts_tablet_manager.cc:1398] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:44.269397 4903 ts_tablet_manager.cc:1398] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:44.270023 4886 tablet_bootstrap.cc:492] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:44.270123 4886 ts_tablet_manager.cc:1367] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.002s
I20240502 14:00:44.273160 4886 raft_consensus.cc:348] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.273305 4886 raft_consensus.cc:374] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.273350 4886 raft_consensus.cc:729] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.273505 4886 consensus_queue.cc:260] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.273864 4886 ts_tablet_manager.cc:1398] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240502 14:00:44.323714 4879 raft_consensus.cc:482] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:44.323901 4879 raft_consensus.cc:504] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.324436 4879 leader_election.cc:288] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:44.325055 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04393d9da74844acbbb13a17c52905b7" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:44.325109 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04393d9da74844acbbb13a17c52905b7" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:44.325194 674 raft_consensus.cc:2454] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:44.325253 541 raft_consensus.cc:2454] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:44.325443 738 leader_election.cc:302] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:44.325593 4879 raft_consensus.cc:2781] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:44.325639 4879 raft_consensus.cc:482] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:44.325678 4879 raft_consensus.cc:3037] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.326823 4879 raft_consensus.cc:504] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.327127 4879 leader_election.cc:288] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:44.327461 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04393d9da74844acbbb13a17c52905b7" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:44.327564 674 raft_consensus.cc:3037] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.327566 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04393d9da74844acbbb13a17c52905b7" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:44.327692 541 raft_consensus.cc:3037] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.328488 674 raft_consensus.cc:2454] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:44.328733 541 raft_consensus.cc:2454] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:44.328984 738 leader_election.cc:302] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:44.329181 4879 raft_consensus.cc:2781] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:44.329286 4879 raft_consensus.cc:686] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:44.329429 4879 consensus_queue.cc:237] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:44.331616 32731 catalog_manager.cc:5484] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:44.334317 4862 raft_consensus.cc:482] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:44.334441 4862 raft_consensus.cc:504] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:44.334954 4862 leader_election.cc:288] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:44.335299 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c73305a96554a31a68e60e2207232fe" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:44.335397 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c73305a96554a31a68e60e2207232fe" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:44.335438 3283 raft_consensus.cc:2454] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:44.335558 409 raft_consensus.cc:2454] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:44.335740 2952 leader_election.cc:302] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:44.335911 4862 raft_consensus.cc:2781] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:44.335974 4862 raft_consensus.cc:482] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:44.336021 4862 raft_consensus.cc:3037] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.337038 4862 raft_consensus.cc:504] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:44.337388 4862 leader_election.cc:288] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:44.337739 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c73305a96554a31a68e60e2207232fe" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:44.337849 3283 raft_consensus.cc:3037] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.337944 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c73305a96554a31a68e60e2207232fe" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:44.338083 409 raft_consensus.cc:3037] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.338773 3283 raft_consensus.cc:2454] T 8c73305a96554a31a68e60e2207232fe P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:44.339083 2952 leader_election.cc:302] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:44.339224 4738 raft_consensus.cc:2781] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:44.339215 409 raft_consensus.cc:2454] T 8c73305a96554a31a68e60e2207232fe P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:44.339407 4738 raft_consensus.cc:686] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:44.339555 4738 consensus_queue.cc:237] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:44.341740 32731 catalog_manager.cc:5484] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:44.443621 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:44.447625 540 raft_consensus.cc:1261] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:44.447676 674 raft_consensus.cc:1261] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:44.448277 4879 consensus_queue.cc:1035] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:44.448501 4634 consensus_queue.cc:1035] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:44.451634 32731 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 04393d9da74844acbbb13a17c52905b7 (table rebalancer_dynamic_table [id=c5040325e91744e28742be8db85f46dd]). Expected version 1 got 0
I20240502 14:00:44.452242 32733 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 04393d9da74844acbbb13a17c52905b7 (table rebalancer_dynamic_table [id=c5040325e91744e28742be8db85f46dd]). Expected version 1 got 0
I20240502 14:00:44.452924 4876 tablet.cc:1722] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:44.453267 4876 tablet.cc:1620] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:44.455988 32726 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 04393d9da74844acbbb13a17c52905b7 (table rebalancer_dynamic_table [id=c5040325e91744e28742be8db85f46dd]). Expected version 1 got 0
I20240502 14:00:44.456558 4914 tablet.cc:1722] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:44.457027 4913 tablet.cc:1722] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:44.457146 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=c5040325e91744e28742be8db85f46dd] alter complete (version 1)
I20240502 14:00:44.457311 4914 tablet.cc:1620] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:44.457423 4913 tablet.cc:1620] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:44.548637 4718 raft_consensus.cc:482] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:44.548764 4718 raft_consensus.cc:504] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:44.549260 4718 leader_election.cc:288] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:44.549671 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e642731d7684e33b82eabf97c380123" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:44.549671 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e642731d7684e33b82eabf97c380123" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:44.549822 3019 raft_consensus.cc:2454] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:44.549822 805 raft_consensus.cc:2454] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:44.550165 2819 leader_election.cc:302] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:44.550587 4718 raft_consensus.cc:2781] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:44.550655 4718 raft_consensus.cc:482] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:44.550695 4718 raft_consensus.cc:3037] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.551717 4718 raft_consensus.cc:504] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:44.552116 4718 leader_election.cc:288] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:44.552736 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e642731d7684e33b82eabf97c380123" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:44.552857 805 raft_consensus.cc:3037] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.553181 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e642731d7684e33b82eabf97c380123" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:44.553311 3019 raft_consensus.cc:3037] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:44.553874 805 raft_consensus.cc:2454] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:44.554437 3019 raft_consensus.cc:2454] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:44.554795 2819 leader_election.cc:302] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:44.555478 4718 raft_consensus.cc:2781] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:44.555676 4718 raft_consensus.cc:686] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:44.555886 4718 consensus_queue.cc:237] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:44.558866 32731 catalog_manager.cc:5484] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:44.564255 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:44.569556 4914 tablet.cc:1722] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:44.569900 4914 tablet.cc:1620] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:44.570093 4913 tablet.cc:1722] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:44.570503 4876 tablet.cc:1722] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:44.570601 4913 tablet.cc:1620] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:44.570776 4876 tablet.cc:1620] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:44.576639 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=c5040325e91744e28742be8db85f46dd] alter complete (version 2)
I20240502 14:00:44.678510 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:44.684482 4914 tablet.cc:1722] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:44.684993 4876 tablet.cc:1722] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:44.687726 4913 tablet.cc:1722] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:44.690040 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=c5040325e91744e28742be8db85f46dd] alter complete (version 3)
I20240502 14:00:44.701947 4194 consensus_queue.cc:1035] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:44.705451 4194 consensus_queue.cc:1035] T 6115d4bcf9e04e7b84e251c8facb2bdf P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:44.792498 32726 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:44.792655 32726 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:44.795210 32726 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 04393d9da74844acbbb13a17c52905b7
I20240502 14:00:44.795794 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 04393d9da74844acbbb13a17c52905b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:44 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:44.795859 784 tablet_service.cc:1514] Processing DeleteTablet for tablet 04393d9da74844acbbb13a17c52905b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:44 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:44.795938 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 04393d9da74844acbbb13a17c52905b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:44 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:44.796164 4922 tablet_replica.cc:330] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:44.796268 4923 tablet_replica.cc:330] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:44.796334 4922 raft_consensus.cc:2229] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:44.796406 4923 raft_consensus.cc:2229] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:44.796514 4921 tablet_replica.cc:330] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:44.796531 4923 raft_consensus.cc:2258] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:44.796550 4922 raft_consensus.cc:2258] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:44.796671 4921 raft_consensus.cc:2229] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:44.796808 4921 raft_consensus.cc:2258] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:44.797353 4923 ts_tablet_manager.cc:1875] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:44.797375 4922 ts_tablet_manager.cc:1875] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:44.797375 4921 ts_tablet_manager.cc:1875] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:44.799469 4923 ts_tablet_manager.cc:1888] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:44.799594 4923 log.cc:1196] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/04393d9da74844acbbb13a17c52905b7
I20240502 14:00:44.799873 4921 ts_tablet_manager.cc:1888] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:44.799896 4922 ts_tablet_manager.cc:1888] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:44.799997 4921 log.cc:1196] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/04393d9da74844acbbb13a17c52905b7
I20240502 14:00:44.800031 4923 ts_tablet_manager.cc:1909] T 04393d9da74844acbbb13a17c52905b7 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:44.800035 4922 log.cc:1196] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/04393d9da74844acbbb13a17c52905b7
I20240502 14:00:44.800338 4921 ts_tablet_manager.cc:1909] T 04393d9da74844acbbb13a17c52905b7 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:44.800392 4922 ts_tablet_manager.cc:1909] T 04393d9da74844acbbb13a17c52905b7 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:44.800851 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 04393d9da74844acbbb13a17c52905b7 (table rebalancer_dynamic_table_new_name [id=c5040325e91744e28742be8db85f46dd]) successfully deleted
I20240502 14:00:44.801043 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 04393d9da74844acbbb13a17c52905b7 (table rebalancer_dynamic_table_new_name [id=c5040325e91744e28742be8db85f46dd]) successfully deleted
I20240502 14:00:44.801270 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 04393d9da74844acbbb13a17c52905b7 (table rebalancer_dynamic_table_new_name [id=c5040325e91744e28742be8db85f46dd]) successfully deleted
I20240502 14:00:44.897334 32726 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:44.901777 4738 consensus_queue.cc:1035] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:44.903146 784 tablet_service.cc:1467] Processing CreateTablet for tablet aebc68feb212424488e3bcfcf804e1f2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b8dcf2bad36e4c9da57c2e4b67e87757]), partition=
I20240502 14:00:44.903357 653 tablet_service.cc:1467] Processing CreateTablet for tablet aebc68feb212424488e3bcfcf804e1f2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b8dcf2bad36e4c9da57c2e4b67e87757]), partition=
I20240502 14:00:44.903434 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aebc68feb212424488e3bcfcf804e1f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.903631 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aebc68feb212424488e3bcfcf804e1f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.904047 3263 tablet_service.cc:1467] Processing CreateTablet for tablet aebc68feb212424488e3bcfcf804e1f2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b8dcf2bad36e4c9da57c2e4b67e87757]), partition=
I20240502 14:00:44.904232 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aebc68feb212424488e3bcfcf804e1f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:44.906036 4927 tablet_bootstrap.cc:492] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:44.906184 4925 tablet_bootstrap.cc:492] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:44.907438 4925 tablet_bootstrap.cc:654] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.907507 4927 tablet_bootstrap.cc:654] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.909394 4927 tablet_bootstrap.cc:492] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:44.909520 4927 ts_tablet_manager.cc:1367] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:44.910084 4927 raft_consensus.cc:348] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:44.910310 4927 raft_consensus.cc:374] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.910404 4927 raft_consensus.cc:729] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.910614 4927 consensus_queue.cc:260] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:44.912010 4925 tablet_bootstrap.cc:492] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:44.912124 4925 ts_tablet_manager.cc:1367] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:44.912262 4927 ts_tablet_manager.cc:1398] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:44.912598 4930 tablet_bootstrap.cc:492] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:44.912626 4925 raft_consensus.cc:348] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:44.912745 4925 raft_consensus.cc:374] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.912791 4925 raft_consensus.cc:729] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.912909 4925 consensus_queue.cc:260] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:44.913228 4925 ts_tablet_manager.cc:1398] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:44.913642 4738 consensus_queue.cc:1035] T 8c73305a96554a31a68e60e2207232fe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:44.917093 4930 tablet_bootstrap.cc:654] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:44.918762 4930 tablet_bootstrap.cc:492] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:44.918849 4930 ts_tablet_manager.cc:1367] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:44.919270 4930 raft_consensus.cc:348] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:44.919425 4930 raft_consensus.cc:374] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:44.919472 4930 raft_consensus.cc:729] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:44.919606 4930 consensus_queue.cc:260] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:44.919963 4930 ts_tablet_manager.cc:1398] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:44.988049 4657 consensus_queue.cc:1035] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:44.991174 4815 consensus_queue.cc:1035] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:45.044318 4634 raft_consensus.cc:482] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:45.044453 4634 raft_consensus.cc:504] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.045025 4634 leader_election.cc:288] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:45.045449 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aebc68feb212424488e3bcfcf804e1f2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:45.045753 3283 raft_consensus.cc:2454] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:45.045893 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aebc68feb212424488e3bcfcf804e1f2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:45.046032 674 raft_consensus.cc:2454] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:45.046094 739 leader_election.cc:302] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:45.046267 4634 raft_consensus.cc:2781] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:45.046339 4634 raft_consensus.cc:482] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:45.046382 4634 raft_consensus.cc:3037] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.047524 4634 raft_consensus.cc:504] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.048002 4634 leader_election.cc:288] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:45.048157 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aebc68feb212424488e3bcfcf804e1f2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:45.048290 3283 raft_consensus.cc:3037] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.048902 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aebc68feb212424488e3bcfcf804e1f2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:45.049019 674 raft_consensus.cc:3037] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.049314 3283 raft_consensus.cc:2454] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:45.049638 739 leader_election.cc:302] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:45.049808 4634 raft_consensus.cc:2781] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:45.049981 4634 raft_consensus.cc:686] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:45.050066 674 raft_consensus.cc:2454] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:45.050134 4634 consensus_queue.cc:237] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.053565 32731 catalog_manager.cc:5484] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:45.088431 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:45.095206 2999 tablet_service.cc:1467] Processing CreateTablet for tablet a3903dc96640461a81aec9c62128b576 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=23424c52c4e640c282457733934fe056]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.095575 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3903dc96640461a81aec9c62128b576. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.095567 3130 tablet_service.cc:1467] Processing CreateTablet for tablet a3903dc96640461a81aec9c62128b576 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=23424c52c4e640c282457733934fe056]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.095652 784 tablet_service.cc:1467] Processing CreateTablet for tablet d60b2e45a34e4d8f95b33e219d75e78f (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=23424c52c4e640c282457733934fe056]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.095847 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3903dc96640461a81aec9c62128b576. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.096028 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d60b2e45a34e4d8f95b33e219d75e78f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.096774 653 tablet_service.cc:1467] Processing CreateTablet for tablet d60b2e45a34e4d8f95b33e219d75e78f (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=23424c52c4e640c282457733934fe056]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.097056 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d60b2e45a34e4d8f95b33e219d75e78f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.097929 4930 tablet_bootstrap.cc:492] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:45.098169 3263 tablet_service.cc:1467] Processing CreateTablet for tablet e750578ccc314e1eaa17b25873cdb3cf (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=23424c52c4e640c282457733934fe056]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.098248 388 tablet_service.cc:1467] Processing CreateTablet for tablet a3903dc96640461a81aec9c62128b576 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=23424c52c4e640c282457733934fe056]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.098464 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e750578ccc314e1eaa17b25873cdb3cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.098534 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3903dc96640461a81aec9c62128b576. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.099179 4930 tablet_bootstrap.cc:654] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.100739 4927 tablet_bootstrap.cc:492] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:45.101059 4925 tablet_bootstrap.cc:492] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:45.101291 4942 tablet_bootstrap.cc:492] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:45.101403 2866 tablet_service.cc:1467] Processing CreateTablet for tablet d60b2e45a34e4d8f95b33e219d75e78f (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=23424c52c4e640c282457733934fe056]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.101687 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d60b2e45a34e4d8f95b33e219d75e78f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.101733 4927 tablet_bootstrap.cc:654] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.102092 4925 tablet_bootstrap.cc:654] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.102160 4942 tablet_bootstrap.cc:654] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.103631 4930 tablet_bootstrap.cc:492] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:45.103731 4930 ts_tablet_manager.cc:1367] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:45.104295 4930 raft_consensus.cc:348] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.104394 784 tablet_service.cc:1467] Processing CreateTablet for tablet e750578ccc314e1eaa17b25873cdb3cf (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=23424c52c4e640c282457733934fe056]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.104518 4947 tablet_bootstrap.cc:492] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:45.104658 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e750578ccc314e1eaa17b25873cdb3cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.104414 4930 raft_consensus.cc:374] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.104867 4930 raft_consensus.cc:729] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.105183 3130 tablet_service.cc:1467] Processing CreateTablet for tablet e750578ccc314e1eaa17b25873cdb3cf (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=23424c52c4e640c282457733934fe056]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.105446 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e750578ccc314e1eaa17b25873cdb3cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.105557 4947 tablet_bootstrap.cc:654] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.105590 4940 tablet_bootstrap.cc:492] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:45.105778 4930 consensus_queue.cc:260] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.106133 4930 ts_tablet_manager.cc:1398] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:45.106912 4940 tablet_bootstrap.cc:654] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.107721 4943 tablet_bootstrap.cc:492] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:45.108225 4927 tablet_bootstrap.cc:492] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:45.108326 4927 ts_tablet_manager.cc:1367] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:45.108845 4927 raft_consensus.cc:348] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.108992 4943 tablet_bootstrap.cc:654] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.108991 4927 raft_consensus.cc:374] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.109026 4927 raft_consensus.cc:729] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.109146 4927 consensus_queue.cc:260] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.109500 4927 ts_tablet_manager.cc:1398] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.110543 4930 tablet_bootstrap.cc:492] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:45.111524 4930 tablet_bootstrap.cc:654] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.111893 4925 tablet_bootstrap.cc:492] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:45.111987 4925 ts_tablet_manager.cc:1367] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:00:45.112147 4942 tablet_bootstrap.cc:492] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:45.112253 4942 ts_tablet_manager.cc:1367] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:00:45.112504 4925 raft_consensus.cc:348] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.112633 4925 raft_consensus.cc:374] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.112668 4925 raft_consensus.cc:729] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.112756 4942 raft_consensus.cc:348] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.112795 4925 consensus_queue.cc:260] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.112865 4942 raft_consensus.cc:374] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.113014 4942 raft_consensus.cc:729] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.113124 4925 ts_tablet_manager.cc:1398] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.113178 4942 consensus_queue.cc:260] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.113710 4942 ts_tablet_manager.cc:1398] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:45.115103 4930 tablet_bootstrap.cc:492] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:45.115185 4930 ts_tablet_manager.cc:1367] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:45.115628 4930 raft_consensus.cc:348] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.115738 4930 raft_consensus.cc:374] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.115773 4930 raft_consensus.cc:729] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.115900 4930 consensus_queue.cc:260] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.116178 4930 ts_tablet_manager.cc:1398] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.116868 4943 tablet_bootstrap.cc:492] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:45.116993 4943 ts_tablet_manager.cc:1367] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:45.118074 4947 tablet_bootstrap.cc:492] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:45.118121 4943 raft_consensus.cc:348] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.118188 4947 ts_tablet_manager.cc:1367] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 14:00:45.118239 4943 raft_consensus.cc:374] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.118428 4943 raft_consensus.cc:729] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.118588 4943 consensus_queue.cc:260] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.118691 4947 raft_consensus.cc:348] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.118867 4947 raft_consensus.cc:374] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.118949 4943 ts_tablet_manager.cc:1398] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:45.118948 4947 raft_consensus.cc:729] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.119139 4947 consensus_queue.cc:260] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.119370 4943 tablet_bootstrap.cc:492] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:45.120460 4943 tablet_bootstrap.cc:654] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.120694 4947 ts_tablet_manager.cc:1398] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:45.121603 4940 tablet_bootstrap.cc:492] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:45.121728 4940 ts_tablet_manager.cc:1367] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 14:00:45.121745 4943 tablet_bootstrap.cc:492] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:45.121914 4943 ts_tablet_manager.cc:1367] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:45.122238 4940 raft_consensus.cc:348] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.122366 4940 raft_consensus.cc:374] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.122412 4940 raft_consensus.cc:729] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.122368 4943 raft_consensus.cc:348] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.122476 4943 raft_consensus.cc:374] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.122527 4940 consensus_queue.cc:260] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.122818 4940 ts_tablet_manager.cc:1398] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.122510 4943 raft_consensus.cc:729] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.123440 4943 consensus_queue.cc:260] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.123971 4943 ts_tablet_manager.cc:1398] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:45.160658 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:45.164574 674 raft_consensus.cc:1261] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:45.164574 3284 raft_consensus.cc:1261] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:45.165010 4634 consensus_queue.cc:1035] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:45.165143 4879 consensus_queue.cc:1035] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:45.168192 4914 tablet.cc:1722] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:45.168330 32732 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet aebc68feb212424488e3bcfcf804e1f2 (table rebalancer_dynamic_table [id=b8dcf2bad36e4c9da57c2e4b67e87757]). Expected version 1 got 0
I20240502 14:00:45.168532 4914 tablet.cc:1620] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:45.169188 4955 tablet.cc:1722] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:45.169422 32726 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet aebc68feb212424488e3bcfcf804e1f2 (table rebalancer_dynamic_table [id=b8dcf2bad36e4c9da57c2e4b67e87757]). Expected version 1 got 0
I20240502 14:00:45.169482 4955 tablet.cc:1620] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:45.172334 4876 tablet.cc:1722] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:45.172646 4876 tablet.cc:1620] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:45.174472 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=b8dcf2bad36e4c9da57c2e4b67e87757] alter complete (version 1)
I20240502 14:00:45.174726 4814 raft_consensus.cc:482] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:45.174865 4814 raft_consensus.cc:504] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.175276 4814 leader_election.cc:288] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:45.175608 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3903dc96640461a81aec9c62128b576" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:45.175748 3019 raft_consensus.cc:2454] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:45.176080 3084 leader_election.cc:302] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:45.176110 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3903dc96640461a81aec9c62128b576" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:45.176234 405 raft_consensus.cc:2454] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:45.176640 4814 raft_consensus.cc:2781] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:45.176705 4814 raft_consensus.cc:482] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:45.176743 4814 raft_consensus.cc:3037] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.177722 4814 raft_consensus.cc:504] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.178025 4814 leader_election.cc:288] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:45.178303 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3903dc96640461a81aec9c62128b576" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:45.178308 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3903dc96640461a81aec9c62128b576" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:45.178416 405 raft_consensus.cc:3037] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.178418 3019 raft_consensus.cc:3037] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.179432 405 raft_consensus.cc:2454] T a3903dc96640461a81aec9c62128b576 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:45.179497 3019 raft_consensus.cc:2454] T a3903dc96640461a81aec9c62128b576 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:45.179805 3086 leader_election.cc:302] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:45.179984 4814 raft_consensus.cc:2781] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:45.180084 4814 raft_consensus.cc:686] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:45.180217 4814 consensus_queue.cc:237] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.182641 32732 catalog_manager.cc:5484] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:45.193490 4637 raft_consensus.cc:482] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:45.193614 4637 raft_consensus.cc:504] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.193985 4637 leader_election.cc:288] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:45.194254 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e750578ccc314e1eaa17b25873cdb3cf" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:45.194388 805 raft_consensus.cc:2454] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:45.194396 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e750578ccc314e1eaa17b25873cdb3cf" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:45.194530 3150 raft_consensus.cc:2454] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:45.194801 3219 leader_election.cc:302] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:45.194979 4637 raft_consensus.cc:2781] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:45.195039 4637 raft_consensus.cc:482] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:45.195078 4637 raft_consensus.cc:3037] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.196156 4637 raft_consensus.cc:504] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.196468 4637 leader_election.cc:288] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:45.196851 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e750578ccc314e1eaa17b25873cdb3cf" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:45.196972 805 raft_consensus.cc:3037] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.197055 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e750578ccc314e1eaa17b25873cdb3cf" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:45.197170 3150 raft_consensus.cc:3037] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.198024 805 raft_consensus.cc:2454] T e750578ccc314e1eaa17b25873cdb3cf P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:45.198204 3150 raft_consensus.cc:2454] T e750578ccc314e1eaa17b25873cdb3cf P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:45.198300 3219 leader_election.cc:302] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:45.198576 4637 raft_consensus.cc:2781] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:45.198669 4637 raft_consensus.cc:686] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:45.198858 4637 consensus_queue.cc:237] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:45.200737 4826 raft_consensus.cc:482] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:45.200834 4826 raft_consensus.cc:504] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.201267 4826 leader_election.cc:288] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:45.201320 32732 catalog_manager.cc:5484] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:45.201673 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d60b2e45a34e4d8f95b33e219d75e78f" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:45.201795 2884 raft_consensus.cc:2454] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:45.201959 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d60b2e45a34e4d8f95b33e219d75e78f" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:45.202076 805 raft_consensus.cc:2454] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:45.202068 609 leader_election.cc:302] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:45.202219 4826 raft_consensus.cc:2781] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:45.202265 4826 raft_consensus.cc:482] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:45.202297 4826 raft_consensus.cc:3037] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.203289 4826 raft_consensus.cc:504] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.203717 4826 leader_election.cc:288] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:45.204092 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d60b2e45a34e4d8f95b33e219d75e78f" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:45.204208 2884 raft_consensus.cc:3037] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.204200 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d60b2e45a34e4d8f95b33e219d75e78f" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:45.204311 805 raft_consensus.cc:3037] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.205094 2884 raft_consensus.cc:2454] T d60b2e45a34e4d8f95b33e219d75e78f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:45.205296 805 raft_consensus.cc:2454] T d60b2e45a34e4d8f95b33e219d75e78f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:45.205386 609 leader_election.cc:302] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:45.205700 4826 raft_consensus.cc:2781] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:45.205778 4826 raft_consensus.cc:686] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:45.205937 4826 consensus_queue.cc:237] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.207970 32732 catalog_manager.cc:5484] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:45.282613 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:45.288045 4914 tablet.cc:1722] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:45.288420 4914 tablet.cc:1620] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:45.288417 4955 tablet.cc:1722] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:45.288738 4955 tablet.cc:1620] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:45.290853 4876 tablet.cc:1722] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:45.291193 4876 tablet.cc:1620] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:45.294670 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=b8dcf2bad36e4c9da57c2e4b67e87757] alter complete (version 2)
I20240502 14:00:45.396283 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:45.401484 4914 tablet.cc:1722] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:45.401993 4955 tablet.cc:1722] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:45.403084 4876 tablet.cc:1722] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:45.407182 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=b8dcf2bad36e4c9da57c2e4b67e87757] alter complete (version 3)
I20240502 14:00:45.508863 32726 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:45.509085 32726 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:45.511389 32726 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet aebc68feb212424488e3bcfcf804e1f2
I20240502 14:00:45.511936 784 tablet_service.cc:1514] Processing DeleteTablet for tablet aebc68feb212424488e3bcfcf804e1f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:45 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:45.512261 4961 tablet_replica.cc:330] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:45.512301 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet aebc68feb212424488e3bcfcf804e1f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:45 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:45.512446 4961 raft_consensus.cc:2229] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:45.512599 4962 tablet_replica.cc:330] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:45.512719 4961 raft_consensus.cc:2258] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:45.512785 4962 raft_consensus.cc:2229] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:45.512964 4962 raft_consensus.cc:2258] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:45.513422 653 tablet_service.cc:1514] Processing DeleteTablet for tablet aebc68feb212424488e3bcfcf804e1f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:45 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:45.513569 4961 ts_tablet_manager.cc:1875] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:45.513796 4963 tablet_replica.cc:330] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:45.513761 4962 ts_tablet_manager.cc:1875] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:45.514055 4963 raft_consensus.cc:2229] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:45.514338 4963 raft_consensus.cc:2258] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:45.515026 4963 ts_tablet_manager.cc:1875] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:45.515998 4961 ts_tablet_manager.cc:1888] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:45.516062 4962 ts_tablet_manager.cc:1888] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:45.516085 4961 log.cc:1196] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/aebc68feb212424488e3bcfcf804e1f2
I20240502 14:00:45.516160 4962 log.cc:1196] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/aebc68feb212424488e3bcfcf804e1f2
I20240502 14:00:45.516430 4962 ts_tablet_manager.cc:1909] T aebc68feb212424488e3bcfcf804e1f2 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:45.516420 4961 ts_tablet_manager.cc:1909] T aebc68feb212424488e3bcfcf804e1f2 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:45.517072 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet aebc68feb212424488e3bcfcf804e1f2 (table rebalancer_dynamic_table_new_name [id=b8dcf2bad36e4c9da57c2e4b67e87757]) successfully deleted
I20240502 14:00:45.517072 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet aebc68feb212424488e3bcfcf804e1f2 (table rebalancer_dynamic_table_new_name [id=b8dcf2bad36e4c9da57c2e4b67e87757]) successfully deleted
I20240502 14:00:45.517392 4963 ts_tablet_manager.cc:1888] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:45.517460 4963 log.cc:1196] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/aebc68feb212424488e3bcfcf804e1f2
I20240502 14:00:45.517684 4963 ts_tablet_manager.cc:1909] T aebc68feb212424488e3bcfcf804e1f2 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:45.518476 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet aebc68feb212424488e3bcfcf804e1f2 (table rebalancer_dynamic_table_new_name [id=b8dcf2bad36e4c9da57c2e4b67e87757]) successfully deleted
I20240502 14:00:45.615571 32726 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:45.621412 388 tablet_service.cc:1467] Processing CreateTablet for tablet d98d53a8671e49b19e9b705c715d6f5d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f0660bb0331440e0af0847cd8b796b9b]), partition=
I20240502 14:00:45.621598 784 tablet_service.cc:1467] Processing CreateTablet for tablet d98d53a8671e49b19e9b705c715d6f5d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f0660bb0331440e0af0847cd8b796b9b]), partition=
I20240502 14:00:45.621721 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d98d53a8671e49b19e9b705c715d6f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.621817 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d98d53a8671e49b19e9b705c715d6f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.624620 4966 tablet_bootstrap.cc:492] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:45.624634 2999 tablet_service.cc:1467] Processing CreateTablet for tablet d98d53a8671e49b19e9b705c715d6f5d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f0660bb0331440e0af0847cd8b796b9b]), partition=
I20240502 14:00:45.624897 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d98d53a8671e49b19e9b705c715d6f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.625674 4966 tablet_bootstrap.cc:654] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.626400 4965 tablet_bootstrap.cc:492] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:45.627012 4966 tablet_bootstrap.cc:492] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:45.627113 4966 ts_tablet_manager.cc:1367] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:45.627313 4968 tablet_bootstrap.cc:492] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:45.627588 4966 raft_consensus.cc:348] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:45.627712 4966 raft_consensus.cc:374] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.627792 4966 raft_consensus.cc:729] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.627966 4966 consensus_queue.cc:260] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:45.628342 4966 ts_tablet_manager.cc:1398] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:45.628417 4968 tablet_bootstrap.cc:654] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.630035 4965 tablet_bootstrap.cc:654] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.630132 4968 tablet_bootstrap.cc:492] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:45.630262 4968 ts_tablet_manager.cc:1367] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:45.630781 4968 raft_consensus.cc:348] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:45.630961 4968 raft_consensus.cc:374] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.631049 4968 raft_consensus.cc:729] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.631220 4968 consensus_queue.cc:260] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:45.631814 4968 ts_tablet_manager.cc:1398] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.633816 4965 tablet_bootstrap.cc:492] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:45.634057 4965 ts_tablet_manager.cc:1367] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:45.634546 4965 raft_consensus.cc:348] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:45.634691 4965 raft_consensus.cc:374] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.634757 4965 raft_consensus.cc:729] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.634932 4965 consensus_queue.cc:260] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:45.635268 4965 ts_tablet_manager.cc:1398] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.641023 4194 consensus_queue.cc:1035] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:45.644449 4814 consensus_queue.cc:1035] T a3903dc96640461a81aec9c62128b576 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:45.678202 4826 consensus_queue.cc:1035] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:45.681459 4826 consensus_queue.cc:1035] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:45.719135 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:45.729050 653 tablet_service.cc:1467] Processing CreateTablet for tablet fee3c05d88514648b8291e01eda2db2b (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4bef3bba6d9447d8a24755864aec22c2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.729398 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee3c05d88514648b8291e01eda2db2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.729398 2866 tablet_service.cc:1467] Processing CreateTablet for tablet fee3c05d88514648b8291e01eda2db2b (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4bef3bba6d9447d8a24755864aec22c2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.729398 3263 tablet_service.cc:1467] Processing CreateTablet for tablet f51e005446c64a07b18f0dbf5056afbc (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4bef3bba6d9447d8a24755864aec22c2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.729674 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f51e005446c64a07b18f0dbf5056afbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.729674 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee3c05d88514648b8291e01eda2db2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.731657 4980 tablet_bootstrap.cc:492] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:45.732196 520 tablet_service.cc:1467] Processing CreateTablet for tablet fee3c05d88514648b8291e01eda2db2b (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4bef3bba6d9447d8a24755864aec22c2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.732484 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee3c05d88514648b8291e01eda2db2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.732506 4981 tablet_bootstrap.cc:492] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:45.732486 654 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9fc8aa46794f459538e925564e0693 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4bef3bba6d9447d8a24755864aec22c2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.732712 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9fc8aa46794f459538e925564e0693. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.732906 4980 tablet_bootstrap.cc:654] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.733451 4981 tablet_bootstrap.cc:654] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.734571 4985 tablet_bootstrap.cc:492] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:45.735556 2866 tablet_service.cc:1467] Processing CreateTablet for tablet f51e005446c64a07b18f0dbf5056afbc (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4bef3bba6d9447d8a24755864aec22c2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.735808 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f51e005446c64a07b18f0dbf5056afbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.735890 4981 tablet_bootstrap.cc:492] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:45.735780 784 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9fc8aa46794f459538e925564e0693 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4bef3bba6d9447d8a24755864aec22c2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.736023 4981 ts_tablet_manager.cc:1367] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:45.736029 4985 tablet_bootstrap.cc:654] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.736145 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9fc8aa46794f459538e925564e0693. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.736454 4981 raft_consensus.cc:348] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.736558 4981 raft_consensus.cc:374] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.736588 4981 raft_consensus.cc:729] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.736744 4981 consensus_queue.cc:260] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.737869 4981 ts_tablet_manager.cc:1398] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:45.738297 4966 tablet_bootstrap.cc:492] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:45.738216 4982 tablet_bootstrap.cc:492] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:45.739233 4966 tablet_bootstrap.cc:654] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.740131 388 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9fc8aa46794f459538e925564e0693 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4bef3bba6d9447d8a24755864aec22c2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.740229 4982 tablet_bootstrap.cc:654] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.740438 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9fc8aa46794f459538e925564e0693. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.740542 4966 tablet_bootstrap.cc:492] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:45.740628 4966 ts_tablet_manager.cc:1367] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:45.741014 653 tablet_service.cc:1467] Processing CreateTablet for tablet f51e005446c64a07b18f0dbf5056afbc (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=4bef3bba6d9447d8a24755864aec22c2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:45.741164 4966 raft_consensus.cc:348] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:45.741246 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f51e005446c64a07b18f0dbf5056afbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:45.741278 4966 raft_consensus.cc:374] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.741308 4982 tablet_bootstrap.cc:492] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:45.741312 4966 raft_consensus.cc:729] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.741381 4982 ts_tablet_manager.cc:1367] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:45.741420 4966 consensus_queue.cc:260] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:45.741701 4966 ts_tablet_manager.cc:1398] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.741816 4982 raft_consensus.cc:348] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:45.741909 4982 raft_consensus.cc:374] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.741942 4982 raft_consensus.cc:729] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.742051 4982 consensus_queue.cc:260] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:45.742293 4982 ts_tablet_manager.cc:1398] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.742447 4982 tablet_bootstrap.cc:492] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:45.742790 4965 tablet_bootstrap.cc:492] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:45.743422 4982 tablet_bootstrap.cc:654] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.743773 4965 tablet_bootstrap.cc:654] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.745131 4982 tablet_bootstrap.cc:492] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:45.745226 4982 ts_tablet_manager.cc:1367] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:45.745255 4980 tablet_bootstrap.cc:492] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:45.745337 4980 ts_tablet_manager.cc:1367] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 14:00:45.745677 4982 raft_consensus.cc:348] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.745787 4982 raft_consensus.cc:374] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.745818 4982 raft_consensus.cc:729] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.745860 4980 raft_consensus.cc:348] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:45.745920 4982 consensus_queue.cc:260] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.745971 4980 raft_consensus.cc:374] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.746008 4980 raft_consensus.cc:729] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.746177 4982 ts_tablet_manager.cc:1398] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.746130 4980 consensus_queue.cc:260] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:45.746480 4980 ts_tablet_manager.cc:1398] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.746651 4980 tablet_bootstrap.cc:492] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:45.747715 4980 tablet_bootstrap.cc:654] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.749641 4985 tablet_bootstrap.cc:492] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:45.749785 4985 ts_tablet_manager.cc:1367] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:00:45.750267 4965 tablet_bootstrap.cc:492] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:45.750365 4965 ts_tablet_manager.cc:1367] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:45.750315 4985 raft_consensus.cc:348] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:45.750562 4985 raft_consensus.cc:374] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.750604 4985 raft_consensus.cc:729] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.750721 4985 consensus_queue.cc:260] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:45.750852 4965 raft_consensus.cc:348] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:45.750946 4965 raft_consensus.cc:374] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.750975 4965 raft_consensus.cc:729] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.751094 4965 consensus_queue.cc:260] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:45.751145 4985 ts_tablet_manager.cc:1398] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.751364 4965 ts_tablet_manager.cc:1398] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.756280 4637 consensus_queue.cc:1035] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:45.757139 4980 tablet_bootstrap.cc:492] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:45.757239 4980 ts_tablet_manager.cc:1367] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:00:45.757594 4634 raft_consensus.cc:482] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:45.757678 4634 raft_consensus.cc:504] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:45.757751 4980 raft_consensus.cc:348] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:45.757849 4980 raft_consensus.cc:374] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.757880 4980 raft_consensus.cc:729] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.757984 4980 consensus_queue.cc:260] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:45.758092 4634 leader_election.cc:288] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:45.758365 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d98d53a8671e49b19e9b705c715d6f5d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:45.758493 405 raft_consensus.cc:2454] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:45.758469 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d98d53a8671e49b19e9b705c715d6f5d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:45.758589 3019 raft_consensus.cc:2454] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:45.758827 740 leader_election.cc:302] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:45.759104 4634 raft_consensus.cc:2781] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:45.759162 4634 raft_consensus.cc:482] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:45.759207 4634 raft_consensus.cc:3037] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.760361 4634 raft_consensus.cc:504] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:45.760681 4634 leader_election.cc:288] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:45.760707 4916 consensus_queue.cc:1035] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:45.761188 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d98d53a8671e49b19e9b705c715d6f5d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:45.761307 405 raft_consensus.cc:3037] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.761334 4980 ts_tablet_manager.cc:1398] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:45.761529 4980 tablet_bootstrap.cc:492] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:45.762331 405 raft_consensus.cc:2454] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:45.762663 740 leader_election.cc:302] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:45.762882 4634 raft_consensus.cc:2781] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:45.762898 4980 tablet_bootstrap.cc:654] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:45.762961 4634 raft_consensus.cc:686] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:45.763098 4634 consensus_queue.cc:237] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:45.764321 4980 tablet_bootstrap.cc:492] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:45.764410 4980 ts_tablet_manager.cc:1367] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:45.764851 4980 raft_consensus.cc:348] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.764966 4980 raft_consensus.cc:374] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:45.765007 4980 raft_consensus.cc:729] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:45.765122 4980 consensus_queue.cc:260] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.765571 4980 ts_tablet_manager.cc:1398] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.765864 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d98d53a8671e49b19e9b705c715d6f5d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:45.766003 3019 raft_consensus.cc:3037] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.766947 3019 raft_consensus.cc:2454] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:45.768496 32733 catalog_manager.cc:5484] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:45.774567 4634 raft_consensus.cc:482] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:45.774708 4634 raft_consensus.cc:504] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:45.775738 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9fc8aa46794f459538e925564e0693" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:45.775836 4634 leader_election.cc:288] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:45.775918 405 raft_consensus.cc:2454] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:45.776266 740 leader_election.cc:302] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:45.776546 4634 raft_consensus.cc:2781] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:45.776597 4634 raft_consensus.cc:482] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:45.776639 4634 raft_consensus.cc:3037] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.776587 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9fc8aa46794f459538e925564e0693" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:45.776737 674 raft_consensus.cc:2454] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:45.777755 4634 raft_consensus.cc:504] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:45.778071 4634 leader_election.cc:288] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:45.778380 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9fc8aa46794f459538e925564e0693" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:45.778409 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9fc8aa46794f459538e925564e0693" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:45.778484 674 raft_consensus.cc:3037] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.778515 405 raft_consensus.cc:3037] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.779299 405 raft_consensus.cc:2454] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:45.779353 674 raft_consensus.cc:2454] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:45.779596 738 leader_election.cc:302] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:45.779769 4634 raft_consensus.cc:2781] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:45.779955 4634 raft_consensus.cc:686] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:45.780094 4634 consensus_queue.cc:237] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:45.782284 32732 catalog_manager.cc:5484] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:45.804500 4994 raft_consensus.cc:482] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:45.804623 4994 raft_consensus.cc:504] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.805106 4994 leader_election.cc:288] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:45.805429 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f51e005446c64a07b18f0dbf5056afbc" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:45.805474 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f51e005446c64a07b18f0dbf5056afbc" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:45.805569 2886 raft_consensus.cc:2454] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:45.805627 674 raft_consensus.cc:2454] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:45.805866 3219 leader_election.cc:302] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:45.806157 4994 raft_consensus.cc:2781] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:45.806221 4994 raft_consensus.cc:482] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:45.806263 4994 raft_consensus.cc:3037] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.807215 4994 raft_consensus.cc:504] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.807555 4994 leader_election.cc:288] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:45.807951 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f51e005446c64a07b18f0dbf5056afbc" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:45.808059 674 raft_consensus.cc:3037] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.808827 674 raft_consensus.cc:2454] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:45.809067 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f51e005446c64a07b18f0dbf5056afbc" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:45.809098 3216 leader_election.cc:302] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:45.809211 2886 raft_consensus.cc:3037] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.810029 2886 raft_consensus.cc:2454] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:45.810209 4994 raft_consensus.cc:2781] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:45.810324 4994 raft_consensus.cc:686] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:45.810464 4994 consensus_queue.cc:237] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:45.812834 32732 catalog_manager.cc:5484] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:45.820492 4833 raft_consensus.cc:482] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:45.820617 4833 raft_consensus.cc:504] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:45.821154 4833 leader_election.cc:288] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:45.821506 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee3c05d88514648b8291e01eda2db2b" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:45.821574 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee3c05d88514648b8291e01eda2db2b" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:45.821643 2886 raft_consensus.cc:2454] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:45.821686 672 raft_consensus.cc:2454] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:45.822010 477 leader_election.cc:302] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:45.822239 4833 raft_consensus.cc:2781] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:45.822295 4833 raft_consensus.cc:482] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:45.822340 4833 raft_consensus.cc:3037] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.823225 4833 raft_consensus.cc:504] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:45.823551 4833 leader_election.cc:288] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:45.824081 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee3c05d88514648b8291e01eda2db2b" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:45.824087 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fee3c05d88514648b8291e01eda2db2b" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:45.824204 672 raft_consensus.cc:3037] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.824246 2884 raft_consensus.cc:3037] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:45.825104 2884 raft_consensus.cc:2454] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:45.825268 672 raft_consensus.cc:2454] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:45.825382 477 leader_election.cc:302] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:45.825531 4964 raft_consensus.cc:2781] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:45.825611 4964 raft_consensus.cc:686] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:45.825773 4964 consensus_queue.cc:237] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:45.827983 32732 catalog_manager.cc:5484] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:45.885078 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:45.889371 405 raft_consensus.cc:1261] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:45.889478 3019 raft_consensus.cc:1261] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:45.889885 4634 consensus_queue.cc:1035] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:45.890093 4879 consensus_queue.cc:1035] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:45.892709 4914 tablet.cc:1722] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:45.893059 4914 tablet.cc:1620] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:45.893548 5004 tablet.cc:1722] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:45.893626 4976 mvcc.cc:205] Tried to move back new op lower bound from 7023240994359283712 to 7023240993847083008. Current Snapshot: MvccSnapshot[applied={T|T < 7023240994359283712}]
I20240502 14:00:45.893966 5004 tablet.cc:1620] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:45.894778 32726 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet d98d53a8671e49b19e9b705c715d6f5d (table rebalancer_dynamic_table [id=f0660bb0331440e0af0847cd8b796b9b]). Expected version 1 got 0
I20240502 14:00:45.896836 32733 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet d98d53a8671e49b19e9b705c715d6f5d (table rebalancer_dynamic_table [id=f0660bb0331440e0af0847cd8b796b9b]). Expected version 1 got 0
I20240502 14:00:45.897086 5005 tablet.cc:1722] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:45.899194 32732 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet d98d53a8671e49b19e9b705c715d6f5d (table rebalancer_dynamic_table [id=f0660bb0331440e0af0847cd8b796b9b]). Expected version 1 got 0
I20240502 14:00:45.900631 32733 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet d98d53a8671e49b19e9b705c715d6f5d (table rebalancer_dynamic_table [id=f0660bb0331440e0af0847cd8b796b9b]). Expected version 1 got 0
I20240502 14:00:45.900733 5005 tablet.cc:1620] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:45.901254 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=f0660bb0331440e0af0847cd8b796b9b] alter complete (version 1)
I20240502 14:00:46.003193 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:46.011520 4914 tablet.cc:1722] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:46.011914 4914 tablet.cc:1620] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:46.012879 5004 tablet.cc:1722] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:46.013144 5004 tablet.cc:1620] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:46.020562 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=f0660bb0331440e0af0847cd8b796b9b] alter complete (version 2)
I20240502 14:00:46.022274 5005 tablet.cc:1722] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:46.022595 5005 tablet.cc:1620] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:46.128078 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:46.133741 4914 tablet.cc:1722] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:46.136086 5004 tablet.cc:1722] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:46.138139 5005 tablet.cc:1722] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:46.140322 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=f0660bb0331440e0af0847cd8b796b9b] alter complete (version 3)
I20240502 14:00:46.167770 4634 consensus_queue.cc:1035] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:46.170964 5006 consensus_queue.cc:1035] T 3c9fc8aa46794f459538e925564e0693 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:46.201521 4994 consensus_queue.cc:1035] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:46.204497 4916 consensus_queue.cc:1035] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:46.242336 32733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:46.242499 32733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:46.246899 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet d98d53a8671e49b19e9b705c715d6f5d
I20240502 14:00:46.247576 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet d98d53a8671e49b19e9b705c715d6f5d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:46 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:46.247602 388 tablet_service.cc:1514] Processing DeleteTablet for tablet d98d53a8671e49b19e9b705c715d6f5d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:46 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:46.248160 784 tablet_service.cc:1514] Processing DeleteTablet for tablet d98d53a8671e49b19e9b705c715d6f5d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:46 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:46.248301 5012 tablet_replica.cc:330] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:46.248461 5012 raft_consensus.cc:2229] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:46.248672 5012 raft_consensus.cc:2258] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.248754 5014 tablet_replica.cc:330] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:46.248566 5013 tablet_replica.cc:330] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:46.249189 5014 raft_consensus.cc:2229] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:46.249418 5014 raft_consensus.cc:2258] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.249460 5013 raft_consensus.cc:2229] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:46.249747 5013 raft_consensus.cc:2258] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.250521 5013 ts_tablet_manager.cc:1875] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:46.250785 5012 ts_tablet_manager.cc:1875] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:46.252676 5014 ts_tablet_manager.cc:1875] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:46.253331 5013 ts_tablet_manager.cc:1888] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:46.253440 5013 log.cc:1196] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/d98d53a8671e49b19e9b705c715d6f5d
I20240502 14:00:46.253769 5012 ts_tablet_manager.cc:1888] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:46.253832 5013 ts_tablet_manager.cc:1909] T d98d53a8671e49b19e9b705c715d6f5d P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:46.253880 5012 log.cc:1196] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/d98d53a8671e49b19e9b705c715d6f5d
I20240502 14:00:46.254199 5012 ts_tablet_manager.cc:1909] T d98d53a8671e49b19e9b705c715d6f5d P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:46.254619 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet d98d53a8671e49b19e9b705c715d6f5d (table rebalancer_dynamic_table_new_name [id=f0660bb0331440e0af0847cd8b796b9b]) successfully deleted
I20240502 14:00:46.254863 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet d98d53a8671e49b19e9b705c715d6f5d (table rebalancer_dynamic_table_new_name [id=f0660bb0331440e0af0847cd8b796b9b]) successfully deleted
I20240502 14:00:46.255692 5014 ts_tablet_manager.cc:1888] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:46.255769 5014 log.cc:1196] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/d98d53a8671e49b19e9b705c715d6f5d
I20240502 14:00:46.256088 5014 ts_tablet_manager.cc:1909] T d98d53a8671e49b19e9b705c715d6f5d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:46.256922 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet d98d53a8671e49b19e9b705c715d6f5d (table rebalancer_dynamic_table_new_name [id=f0660bb0331440e0af0847cd8b796b9b]) successfully deleted
I20240502 14:00:46.342295 4833 consensus_queue.cc:1035] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:46.346009 4833 consensus_queue.cc:1035] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:46.349089 32726 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:46.349910 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:46.355803 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 31464f5378854fce86c7dd05add0bbba (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d161ae1c060b413bbf93bd4bb12833d6]), partition=
I20240502 14:00:46.355965 520 tablet_service.cc:1467] Processing CreateTablet for tablet 31464f5378854fce86c7dd05add0bbba (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d161ae1c060b413bbf93bd4bb12833d6]), partition=
I20240502 14:00:46.356153 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31464f5378854fce86c7dd05add0bbba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.356261 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31464f5378854fce86c7dd05add0bbba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.358482 5020 tablet_bootstrap.cc:492] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:46.358457 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 31464f5378854fce86c7dd05add0bbba (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d161ae1c060b413bbf93bd4bb12833d6]), partition=
I20240502 14:00:46.358719 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31464f5378854fce86c7dd05add0bbba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.359124 5021 tablet_bootstrap.cc:492] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:46.359421 5020 tablet_bootstrap.cc:654] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.360167 5021 tablet_bootstrap.cc:654] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.361078 5024 tablet_bootstrap.cc:492] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:46.362048 5024 tablet_bootstrap.cc:654] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.363471 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 792bc86bad314a2bbf4bba3001c78894 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=bb7573d3e8ad494cb5dd23a0f882463f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:46.363615 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 9f6b40db906d4d72bae26a007737789b (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=bb7573d3e8ad494cb5dd23a0f882463f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:46.363731 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 792bc86bad314a2bbf4bba3001c78894. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.364391 3130 tablet_service.cc:1467] Processing CreateTablet for tablet fa301acea4ad4902839382b7f690f68f (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=bb7573d3e8ad494cb5dd23a0f882463f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:46.364463 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f6b40db906d4d72bae26a007737789b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.364660 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa301acea4ad4902839382b7f690f68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.364749 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 792bc86bad314a2bbf4bba3001c78894 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=bb7573d3e8ad494cb5dd23a0f882463f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:46.364960 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 792bc86bad314a2bbf4bba3001c78894. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.366297 2866 tablet_service.cc:1467] Processing CreateTablet for tablet fa301acea4ad4902839382b7f690f68f (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=bb7573d3e8ad494cb5dd23a0f882463f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:46.366575 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa301acea4ad4902839382b7f690f68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.367122 786 tablet_service.cc:1467] Processing CreateTablet for tablet fa301acea4ad4902839382b7f690f68f (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=bb7573d3e8ad494cb5dd23a0f882463f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:46.367369 786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa301acea4ad4902839382b7f690f68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.368592 5024 tablet_bootstrap.cc:492] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:46.368641 5020 tablet_bootstrap.cc:492] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:46.368676 5024 ts_tablet_manager.cc:1367] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:46.368721 5020 ts_tablet_manager.cc:1367] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:46.368983 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 9f6b40db906d4d72bae26a007737789b (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=bb7573d3e8ad494cb5dd23a0f882463f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:46.369129 5024 raft_consensus.cc:348] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:46.369129 5020 raft_consensus.cc:348] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:46.369215 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f6b40db906d4d72bae26a007737789b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.369231 5024 raft_consensus.cc:374] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.369231 5020 raft_consensus.cc:374] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.369266 5020 raft_consensus.cc:729] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.369266 5024 raft_consensus.cc:729] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.369395 5020 consensus_queue.cc:260] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:46.369418 5024 consensus_queue.cc:260] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:46.369709 5020 ts_tablet_manager.cc:1398] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:46.371434 784 tablet_service.cc:1467] Processing CreateTablet for tablet 792bc86bad314a2bbf4bba3001c78894 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=bb7573d3e8ad494cb5dd23a0f882463f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:46.371682 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 792bc86bad314a2bbf4bba3001c78894. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.371850 5024 ts_tablet_manager.cc:1398] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:46.372277 5024 tablet_bootstrap.cc:492] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:46.373155 5024 tablet_bootstrap.cc:654] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.373620 785 tablet_service.cc:1467] Processing CreateTablet for tablet 9f6b40db906d4d72bae26a007737789b (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=bb7573d3e8ad494cb5dd23a0f882463f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:46.373788 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f6b40db906d4d72bae26a007737789b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.374068 5024 tablet_bootstrap.cc:492] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:46.374122 5024 ts_tablet_manager.cc:1367] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:46.374512 5024 raft_consensus.cc:348] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:46.374584 5024 raft_consensus.cc:374] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.374604 5024 raft_consensus.cc:729] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.374703 5024 consensus_queue.cc:260] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:46.374953 5024 ts_tablet_manager.cc:1398] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:46.378188 5021 tablet_bootstrap.cc:492] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:46.378291 5021 ts_tablet_manager.cc:1367] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 14:00:46.378901 5021 raft_consensus.cc:348] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:46.379019 5021 raft_consensus.cc:374] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.379050 5021 raft_consensus.cc:729] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.379165 5021 consensus_queue.cc:260] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:46.380180 5021 ts_tablet_manager.cc:1398] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:46.380369 5021 tablet_bootstrap.cc:492] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:46.381682 5021 tablet_bootstrap.cc:654] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.382334 5027 tablet_bootstrap.cc:492] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:46.382882 5021 tablet_bootstrap.cc:492] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:46.383024 5021 ts_tablet_manager.cc:1367] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:46.383463 5021 raft_consensus.cc:348] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:46.383566 5027 tablet_bootstrap.cc:654] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.383639 5021 raft_consensus.cc:374] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.384316 5028 tablet_bootstrap.cc:492] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:46.385010 5021 raft_consensus.cc:729] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.385185 5021 consensus_queue.cc:260] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:46.385387 5028 tablet_bootstrap.cc:654] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.385531 5021 ts_tablet_manager.cc:1398] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:46.386096 5026 tablet_bootstrap.cc:492] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:46.386307 5021 tablet_bootstrap.cc:492] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:46.387158 5026 tablet_bootstrap.cc:654] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.387539 5021 tablet_bootstrap.cc:654] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.388336 5026 tablet_bootstrap.cc:492] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:46.388424 5026 ts_tablet_manager.cc:1367] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:46.388892 5026 raft_consensus.cc:348] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:46.389000 5026 raft_consensus.cc:374] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.389040 5026 raft_consensus.cc:729] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.389154 5026 consensus_queue.cc:260] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:46.389412 5026 ts_tablet_manager.cc:1398] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:46.389586 5026 tablet_bootstrap.cc:492] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:46.390403 5021 tablet_bootstrap.cc:492] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:46.390535 5027 tablet_bootstrap.cc:492] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:46.390583 5021 ts_tablet_manager.cc:1367] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:46.390594 5026 tablet_bootstrap.cc:654] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.390671 5027 ts_tablet_manager.cc:1367] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:46.390959 5021 raft_consensus.cc:348] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:46.391050 5021 raft_consensus.cc:374] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.391081 5021 raft_consensus.cc:729] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.391218 5021 consensus_queue.cc:260] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:46.391182 5027 raft_consensus.cc:348] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:46.391315 5027 raft_consensus.cc:374] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.391526 5027 raft_consensus.cc:729] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.392226 5021 ts_tablet_manager.cc:1398] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:46.394497 5028 tablet_bootstrap.cc:492] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:46.394614 5028 ts_tablet_manager.cc:1367] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:46.395148 5028 raft_consensus.cc:348] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:46.395275 5028 raft_consensus.cc:374] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.395318 5028 raft_consensus.cc:729] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.395462 5028 consensus_queue.cc:260] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:46.395759 5028 ts_tablet_manager.cc:1398] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:46.395974 5028 tablet_bootstrap.cc:492] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:46.396044 5027 consensus_queue.cc:260] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:46.396886 5028 tablet_bootstrap.cc:654] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.402885 5026 tablet_bootstrap.cc:492] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:46.402977 5026 ts_tablet_manager.cc:1367] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 14:00:46.402976 5027 ts_tablet_manager.cc:1398] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20240502 14:00:46.403417 5026 raft_consensus.cc:348] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:46.403529 5026 raft_consensus.cc:374] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.403558 5026 raft_consensus.cc:729] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.403662 5026 consensus_queue.cc:260] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:46.403964 5026 ts_tablet_manager.cc:1398] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:46.405392 5028 tablet_bootstrap.cc:492] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:46.405491 5028 ts_tablet_manager.cc:1367] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:46.405951 5028 raft_consensus.cc:348] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:46.406060 5028 raft_consensus.cc:374] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.406096 5028 raft_consensus.cc:729] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.406220 5028 consensus_queue.cc:260] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:46.406498 5028 ts_tablet_manager.cc:1398] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:46.406644 5028 tablet_bootstrap.cc:492] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:46.407716 5028 tablet_bootstrap.cc:654] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:46.409089 5028 tablet_bootstrap.cc:492] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:46.409178 5028 ts_tablet_manager.cc:1367] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:46.409580 5028 raft_consensus.cc:348] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:46.409709 5028 raft_consensus.cc:374] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:46.409765 5028 raft_consensus.cc:729] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:46.409891 5028 consensus_queue.cc:260] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:46.410152 5028 ts_tablet_manager.cc:1398] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:46.514710 5038 raft_consensus.cc:482] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:46.514977 5038 raft_consensus.cc:504] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:46.515888 5038 leader_election.cc:288] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:46.515930 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "792bc86bad314a2bbf4bba3001c78894" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:46.516063 3283 raft_consensus.cc:2454] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:46.516031 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "792bc86bad314a2bbf4bba3001c78894" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:46.516184 804 raft_consensus.cc:2454] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:46.516503 2954 leader_election.cc:302] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:46.516672 5038 raft_consensus.cc:2781] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:46.516741 5038 raft_consensus.cc:482] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:46.516779 5038 raft_consensus.cc:3037] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.517915 5038 raft_consensus.cc:504] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:46.518289 5038 leader_election.cc:288] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:46.518558 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "792bc86bad314a2bbf4bba3001c78894" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:46.518585 4879 raft_consensus.cc:482] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:46.518663 804 raft_consensus.cc:3037] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.518677 4879 raft_consensus.cc:504] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:46.518882 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "792bc86bad314a2bbf4bba3001c78894" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:46.518980 3283 raft_consensus.cc:3037] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.519121 4879 leader_election.cc:288] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:46.519596 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa301acea4ad4902839382b7f690f68f" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:46.519680 804 raft_consensus.cc:2454] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:46.519716 3150 raft_consensus.cc:2454] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:46.519986 3283 raft_consensus.cc:2454] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:46.520018 741 leader_election.cc:302] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:46.520012 2954 leader_election.cc:302] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:46.520095 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa301acea4ad4902839382b7f690f68f" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:46.520180 4879 raft_consensus.cc:2781] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:46.520229 2886 raft_consensus.cc:2454] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:46.520237 4879 raft_consensus.cc:482] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:46.520301 4879 raft_consensus.cc:3037] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.520447 5038 raft_consensus.cc:2781] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:46.520529 5038 raft_consensus.cc:686] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:46.520665 5038 consensus_queue.cc:237] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:46.521157 4879 raft_consensus.cc:504] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:46.521517 4879 leader_election.cc:288] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:46.521777 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa301acea4ad4902839382b7f690f68f" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:46.521905 3150 raft_consensus.cc:3037] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.521904 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa301acea4ad4902839382b7f690f68f" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:46.522019 2886 raft_consensus.cc:3037] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.522869 3150 raft_consensus.cc:2454] T fa301acea4ad4902839382b7f690f68f P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:46.522927 2886 raft_consensus.cc:2454] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:46.523173 741 leader_election.cc:302] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:46.523346 4879 raft_consensus.cc:2781] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:46.523433 4879 raft_consensus.cc:686] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:46.523370 32726 catalog_manager.cc:5484] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:46.523589 4879 consensus_queue.cc:237] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:46.526297 32726 catalog_manager.cc:5484] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:46.529469 4916 raft_consensus.cc:482] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:46.529572 4916 raft_consensus.cc:504] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:46.529963 4916 leader_election.cc:288] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:46.530373 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31464f5378854fce86c7dd05add0bbba" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:46.530444 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31464f5378854fce86c7dd05add0bbba" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:46.530509 3019 raft_consensus.cc:2454] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:46.530582 540 raft_consensus.cc:2454] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:46.530808 3216 leader_election.cc:302] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:46.530977 4916 raft_consensus.cc:2781] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:46.531044 4916 raft_consensus.cc:482] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:46.531098 4916 raft_consensus.cc:3037] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.531980 4916 raft_consensus.cc:504] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:46.532284 4916 leader_election.cc:288] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:46.532542 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31464f5378854fce86c7dd05add0bbba" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:46.532649 3019 raft_consensus.cc:3037] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.532685 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31464f5378854fce86c7dd05add0bbba" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:46.532788 540 raft_consensus.cc:3037] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.533586 3019 raft_consensus.cc:2454] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:46.533782 540 raft_consensus.cc:2454] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:46.534209 3216 leader_election.cc:302] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:46.534404 4916 raft_consensus.cc:2781] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:46.534569 4916 raft_consensus.cc:686] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:46.534698 4916 consensus_queue.cc:237] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:46.536661 32726 catalog_manager.cc:5484] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:46.665030 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:46.669394 3019 raft_consensus.cc:1261] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:46.669667 540 raft_consensus.cc:1261] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:46.670320 4916 consensus_queue.cc:1035] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:46.670542 4994 consensus_queue.cc:1035] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:46.673920 32726 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet 31464f5378854fce86c7dd05add0bbba (table rebalancer_dynamic_table [id=d161ae1c060b413bbf93bd4bb12833d6]). Expected version 1 got 0
I20240502 14:00:46.675073 5045 tablet.cc:1722] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:46.675441 5045 tablet.cc:1620] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:46.675720 32733 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 31464f5378854fce86c7dd05add0bbba (table rebalancer_dynamic_table [id=d161ae1c060b413bbf93bd4bb12833d6]). Expected version 1 got 0
I20240502 14:00:46.676352 32726 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet 31464f5378854fce86c7dd05add0bbba (table rebalancer_dynamic_table [id=d161ae1c060b413bbf93bd4bb12833d6]). Expected version 1 got 0
I20240502 14:00:46.677775 5017 mvcc.cc:205] Tried to move back new op lower bound from 7023240997554126848 to 7023240997007187968. Current Snapshot: MvccSnapshot[applied={T|T < 7023240997554126848}]
I20240502 14:00:46.677757 5047 tablet.cc:1722] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:46.678938 32733 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet 31464f5378854fce86c7dd05add0bbba (table rebalancer_dynamic_table [id=d161ae1c060b413bbf93bd4bb12833d6]). Expected version 1 got 0
I20240502 14:00:46.681599 32733 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 31464f5378854fce86c7dd05add0bbba (table rebalancer_dynamic_table [id=d161ae1c060b413bbf93bd4bb12833d6]). Expected version 1 got 0
I20240502 14:00:46.682907 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=d161ae1c060b413bbf93bd4bb12833d6] alter complete (version 1)
I20240502 14:00:46.683292 5047 tablet.cc:1620] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:46.689570 5046 tablet.cc:1722] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:46.689872 5046 tablet.cc:1620] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:46.758117 4916 raft_consensus.cc:482] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:46.758244 4916 raft_consensus.cc:504] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:46.758626 4916 leader_election.cc:288] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:46.758944 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6b40db906d4d72bae26a007737789b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:46.759095 3150 raft_consensus.cc:2454] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:46.759317 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6b40db906d4d72bae26a007737789b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:46.759356 3219 leader_election.cc:302] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:46.759462 804 raft_consensus.cc:2454] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:46.759559 4916 raft_consensus.cc:2781] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:46.759615 4916 raft_consensus.cc:482] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:46.759660 4916 raft_consensus.cc:3037] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.760576 4916 raft_consensus.cc:504] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:46.760859 4916 leader_election.cc:288] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:46.761137 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6b40db906d4d72bae26a007737789b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:46.761231 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6b40db906d4d72bae26a007737789b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:46.761276 804 raft_consensus.cc:3037] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.761346 3150 raft_consensus.cc:3037] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:46.762256 804 raft_consensus.cc:2454] T 9f6b40db906d4d72bae26a007737789b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:46.762256 3150 raft_consensus.cc:2454] T 9f6b40db906d4d72bae26a007737789b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:46.762566 3219 leader_election.cc:302] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:46.762872 4916 raft_consensus.cc:2781] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:46.763275 4916 raft_consensus.cc:686] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:46.763419 4916 consensus_queue.cc:237] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:46.765940 32733 catalog_manager.cc:5484] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:46.785471 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:46.791543 5045 tablet.cc:1722] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:46.791918 5045 tablet.cc:1620] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:46.794420 5047 tablet.cc:1722] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:46.794739 5047 tablet.cc:1620] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:46.794819 5046 tablet.cc:1722] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:46.795071 5046 tablet.cc:1620] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:46.797298 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=d161ae1c060b413bbf93bd4bb12833d6] alter complete (version 2)
I20240502 14:00:46.898866 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:46.904825 5045 tablet.cc:1722] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:46.905522 5047 tablet.cc:1722] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:46.909271 5046 tablet.cc:1722] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:46.910605 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=d161ae1c060b413bbf93bd4bb12833d6] alter complete (version 3)
I20240502 14:00:47.012630 32733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:47.012789 32733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:47.023849 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 31464f5378854fce86c7dd05add0bbba
I20240502 14:00:47.024596 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 31464f5378854fce86c7dd05add0bbba with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:47 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:47.024886 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 31464f5378854fce86c7dd05add0bbba with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:47 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:47.025074 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet 31464f5378854fce86c7dd05add0bbba with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:47 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:47.025358 5054 tablet_replica.cc:330] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:47.025389 5053 tablet_replica.cc:330] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:47.025549 5054 raft_consensus.cc:2229] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:47.025549 5053 raft_consensus.cc:2229] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:47.025676 5053 raft_consensus.cc:2258] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:47.025815 5054 raft_consensus.cc:2258] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:47.026436 5053 ts_tablet_manager.cc:1875] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:47.026436 5054 ts_tablet_manager.cc:1875] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:47.028864 5054 ts_tablet_manager.cc:1888] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:47.028956 5054 log.cc:1196] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/31464f5378854fce86c7dd05add0bbba
I20240502 14:00:47.029259 5054 ts_tablet_manager.cc:1909] T 31464f5378854fce86c7dd05add0bbba P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:47.029997 5053 ts_tablet_manager.cc:1888] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:47.030264 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 31464f5378854fce86c7dd05add0bbba (table rebalancer_dynamic_table_new_name [id=d161ae1c060b413bbf93bd4bb12833d6]) successfully deleted
I20240502 14:00:47.030450 5052 tablet_replica.cc:330] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:47.030576 5053 log.cc:1196] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/31464f5378854fce86c7dd05add0bbba
I20240502 14:00:47.030612 5052 raft_consensus.cc:2229] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:47.030791 5052 raft_consensus.cc:2258] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:47.031481 5052 ts_tablet_manager.cc:1875] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:47.032351 5053 ts_tablet_manager.cc:1909] T 31464f5378854fce86c7dd05add0bbba P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:47.033149 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 31464f5378854fce86c7dd05add0bbba (table rebalancer_dynamic_table_new_name [id=d161ae1c060b413bbf93bd4bb12833d6]) successfully deleted
I20240502 14:00:47.033880 5052 ts_tablet_manager.cc:1888] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:47.033954 5052 log.cc:1196] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/31464f5378854fce86c7dd05add0bbba
I20240502 14:00:47.034248 5052 ts_tablet_manager.cc:1909] T 31464f5378854fce86c7dd05add0bbba P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:47.034896 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 31464f5378854fce86c7dd05add0bbba (table rebalancer_dynamic_table_new_name [id=d161ae1c060b413bbf93bd4bb12833d6]) successfully deleted
I20240502 14:00:47.035298 5006 consensus_queue.cc:1035] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:47.044152 5006 consensus_queue.cc:1035] T fa301acea4ad4902839382b7f690f68f P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:47.050441 5038 consensus_queue.cc:1035] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:47.054234 4738 consensus_queue.cc:1035] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:47.125900 32726 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:47.131444 520 tablet_service.cc:1467] Processing CreateTablet for tablet 9435e5c1425d4b308c70f72ce718db30 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2c8a16f241de4db3bb0d105eb61c8446]), partition=
I20240502 14:00:47.131608 653 tablet_service.cc:1467] Processing CreateTablet for tablet 9435e5c1425d4b308c70f72ce718db30 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2c8a16f241de4db3bb0d105eb61c8446]), partition=
I20240502 14:00:47.131769 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9435e5c1425d4b308c70f72ce718db30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.131846 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9435e5c1425d4b308c70f72ce718db30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.131786 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 9435e5c1425d4b308c70f72ce718db30 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2c8a16f241de4db3bb0d105eb61c8446]), partition=
I20240502 14:00:47.132071 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9435e5c1425d4b308c70f72ce718db30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.134575 5064 tablet_bootstrap.cc:492] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:47.134716 5065 tablet_bootstrap.cc:492] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:47.134799 5063 tablet_bootstrap.cc:492] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:47.135763 5065 tablet_bootstrap.cc:654] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.136309 5064 tablet_bootstrap.cc:654] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.137213 5065 tablet_bootstrap.cc:492] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:47.137318 5065 ts_tablet_manager.cc:1367] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:47.137337 5063 tablet_bootstrap.cc:654] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.137847 5065 raft_consensus.cc:348] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:47.137991 5065 raft_consensus.cc:374] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.138123 5065 raft_consensus.cc:729] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.138289 5064 tablet_bootstrap.cc:492] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:47.138312 5065 consensus_queue.cc:260] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:47.138437 5064 ts_tablet_manager.cc:1367] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:47.138854 5064 raft_consensus.cc:348] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:47.138971 5064 raft_consensus.cc:374] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.139012 5064 raft_consensus.cc:729] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.139142 5064 consensus_queue.cc:260] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:47.139509 5065 ts_tablet_manager.cc:1398] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:47.139544 5064 ts_tablet_manager.cc:1398] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:47.140213 5063 tablet_bootstrap.cc:492] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:47.140316 5063 ts_tablet_manager.cc:1367] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:47.140723 5063 raft_consensus.cc:348] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:47.140875 5063 raft_consensus.cc:374] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.140939 5063 raft_consensus.cc:729] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.141088 5063 consensus_queue.cc:260] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:47.141438 5063 ts_tablet_manager.cc:1398] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:47.212421 4972 raft_consensus.cc:482] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:47.212546 4972 raft_consensus.cc:504] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:47.213014 4972 leader_election.cc:288] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:47.213413 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9435e5c1425d4b308c70f72ce718db30" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:47.213421 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9435e5c1425d4b308c70f72ce718db30" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:47.213567 674 raft_consensus.cc:2454] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:47.213582 540 raft_consensus.cc:2454] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:47.213905 3084 leader_election.cc:302] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:47.214119 4972 raft_consensus.cc:2781] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:47.214174 4972 raft_consensus.cc:482] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:47.214215 4972 raft_consensus.cc:3037] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.215265 4972 raft_consensus.cc:504] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:47.215579 4972 leader_election.cc:288] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:47.215929 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9435e5c1425d4b308c70f72ce718db30" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:47.215950 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9435e5c1425d4b308c70f72ce718db30" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:47.216040 540 raft_consensus.cc:3037] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.216063 674 raft_consensus.cc:3037] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.216820 674 raft_consensus.cc:2454] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:47.216820 540 raft_consensus.cc:2454] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:47.217097 3084 leader_election.cc:302] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:47.217253 4972 raft_consensus.cc:2781] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:47.217350 4972 raft_consensus.cc:686] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:47.217496 4972 consensus_queue.cc:237] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:47.220883 32726 catalog_manager.cc:5484] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:47.235606 4994 consensus_queue.cc:1035] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:47.239190 4994 consensus_queue.cc:1035] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:47.330322 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:47.334522 541 raft_consensus.cc:1261] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:47.334746 674 raft_consensus.cc:1261] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:47.335001 4972 consensus_queue.cc:1035] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240502 14:00:47.335675 4194 consensus_queue.cc:1035] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:47.336024 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:47.340027 32733 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 9435e5c1425d4b308c70f72ce718db30 (table rebalancer_dynamic_table [id=2c8a16f241de4db3bb0d105eb61c8446]). Expected version 1 got 0
I20240502 14:00:47.340780 5077 tablet.cc:1722] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:47.340934 5060 mvcc.cc:205] Tried to move back new op lower bound from 7023241000278228992 to 7023240999804100608. Current Snapshot: MvccSnapshot[applied={T|T < 7023241000278228992}]
I20240502 14:00:47.341128 5077 tablet.cc:1620] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:47.343113 32733 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 9435e5c1425d4b308c70f72ce718db30 (table rebalancer_dynamic_table [id=2c8a16f241de4db3bb0d105eb61c8446]). Expected version 1 got 0
I20240502 14:00:47.344575 5047 tablet.cc:1722] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:47.344913 5047 tablet.cc:1620] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:47.347249 5079 tablet.cc:1722] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:47.347645 32732 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 9435e5c1425d4b308c70f72ce718db30 (table rebalancer_dynamic_table [id=2c8a16f241de4db3bb0d105eb61c8446]). Expected version 1 got 0
I20240502 14:00:47.347774 5079 tablet.cc:1620] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:47.348081 32726 catalog_manager.cc:5576] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887) does not have the latest schema for tablet 9435e5c1425d4b308c70f72ce718db30 (table rebalancer_dynamic_table [id=2c8a16f241de4db3bb0d105eb61c8446]). Expected version 1 got 0
I20240502 14:00:47.352429 785 tablet_service.cc:1467] Processing CreateTablet for tablet 49fa0956a8a74ee087bbeb605942e219 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=5d240759fa884da8bf576a9a2b0ef5e2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:47.352780 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49fa0956a8a74ee087bbeb605942e219. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.352906 388 tablet_service.cc:1467] Processing CreateTablet for tablet 49fa0956a8a74ee087bbeb605942e219 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=5d240759fa884da8bf576a9a2b0ef5e2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:47.353055 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=2c8a16f241de4db3bb0d105eb61c8446] alter complete (version 1)
I20240502 14:00:47.353229 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49fa0956a8a74ee087bbeb605942e219. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.353926 387 tablet_service.cc:1467] Processing CreateTablet for tablet 336053fb7de24efd943708fd39dab990 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=5d240759fa884da8bf576a9a2b0ef5e2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:47.354149 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 336053fb7de24efd943708fd39dab990. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.355304 386 tablet_service.cc:1467] Processing CreateTablet for tablet c47fb9f1470349b3be140e7b70811e7b (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=5d240759fa884da8bf576a9a2b0ef5e2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:47.355544 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c47fb9f1470349b3be140e7b70811e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.355868 2866 tablet_service.cc:1467] Processing CreateTablet for tablet c47fb9f1470349b3be140e7b70811e7b (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=5d240759fa884da8bf576a9a2b0ef5e2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:47.356144 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c47fb9f1470349b3be140e7b70811e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.358080 653 tablet_service.cc:1467] Processing CreateTablet for tablet 49fa0956a8a74ee087bbeb605942e219 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=5d240759fa884da8bf576a9a2b0ef5e2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:47.358366 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49fa0956a8a74ee087bbeb605942e219. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.359300 5081 tablet_bootstrap.cc:492] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:47.359793 2999 tablet_service.cc:1467] Processing CreateTablet for tablet c47fb9f1470349b3be140e7b70811e7b (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=5d240759fa884da8bf576a9a2b0ef5e2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:47.360086 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c47fb9f1470349b3be140e7b70811e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.360495 5081 tablet_bootstrap.cc:654] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.360896 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 336053fb7de24efd943708fd39dab990 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=5d240759fa884da8bf576a9a2b0ef5e2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:47.361207 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 336053fb7de24efd943708fd39dab990. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.362427 5085 tablet_bootstrap.cc:492] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:47.362843 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 336053fb7de24efd943708fd39dab990 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=5d240759fa884da8bf576a9a2b0ef5e2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:47.363119 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 336053fb7de24efd943708fd39dab990. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.363588 5063 tablet_bootstrap.cc:492] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:47.364037 5085 tablet_bootstrap.cc:654] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.365795 5081 tablet_bootstrap.cc:492] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:47.365906 5082 tablet_bootstrap.cc:492] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:47.365906 5081 ts_tablet_manager.cc:1367] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:47.366313 5081 raft_consensus.cc:348] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:47.366537 5081 raft_consensus.cc:374] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.366621 5081 raft_consensus.cc:729] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.366830 5083 tablet_bootstrap.cc:492] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:47.366812 5081 consensus_queue.cc:260] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:47.367420 5063 tablet_bootstrap.cc:654] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.367226 5081 ts_tablet_manager.cc:1398] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:47.367796 5083 tablet_bootstrap.cc:654] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.368034 5085 tablet_bootstrap.cc:492] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:47.368115 5085 ts_tablet_manager.cc:1367] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:47.368541 5063 tablet_bootstrap.cc:492] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:47.368621 5063 ts_tablet_manager.cc:1367] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:47.368798 5082 tablet_bootstrap.cc:654] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.369088 5063 raft_consensus.cc:348] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:47.369201 5063 raft_consensus.cc:374] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.369235 5063 raft_consensus.cc:729] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.369377 5063 consensus_queue.cc:260] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:47.369438 5085 raft_consensus.cc:348] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.369551 5085 raft_consensus.cc:374] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.369588 5085 raft_consensus.cc:729] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.369725 5085 consensus_queue.cc:260] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.370002 5085 ts_tablet_manager.cc:1398] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 14:00:47.370851 5086 tablet_bootstrap.cc:492] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:47.370891 5083 tablet_bootstrap.cc:492] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:47.370962 5083 ts_tablet_manager.cc:1367] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:47.371390 5083 raft_consensus.cc:348] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.371471 5083 raft_consensus.cc:374] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.371491 5083 raft_consensus.cc:729] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.371593 5083 consensus_queue.cc:260] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.371882 5083 ts_tablet_manager.cc:1398] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:47.372012 5083 tablet_bootstrap.cc:492] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:47.372803 5083 tablet_bootstrap.cc:654] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.373642 5082 tablet_bootstrap.cc:492] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:47.373672 5086 tablet_bootstrap.cc:654] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.373729 5082 ts_tablet_manager.cc:1367] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:47.374205 5082 raft_consensus.cc:348] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:47.374351 5082 raft_consensus.cc:374] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.374392 5082 raft_consensus.cc:729] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.374514 5082 consensus_queue.cc:260] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:47.374878 5082 ts_tablet_manager.cc:1398] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:47.374946 5086 tablet_bootstrap.cc:492] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:47.375028 5086 ts_tablet_manager.cc:1367] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:47.375088 5082 tablet_bootstrap.cc:492] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:47.375487 5086 raft_consensus.cc:348] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:47.375594 5086 raft_consensus.cc:374] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.375624 5086 raft_consensus.cc:729] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.375741 5086 consensus_queue.cc:260] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:47.376070 5086 ts_tablet_manager.cc:1398] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:47.377385 5083 tablet_bootstrap.cc:492] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:47.377490 5083 ts_tablet_manager.cc:1367] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:00:47.379114 5082 tablet_bootstrap.cc:654] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.380204 5063 ts_tablet_manager.cc:1398] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20240502 14:00:47.381342 5083 raft_consensus.cc:348] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:47.381505 5083 raft_consensus.cc:374] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.381578 5083 raft_consensus.cc:729] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.381713 5083 consensus_queue.cc:260] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:47.383134 5083 ts_tablet_manager.cc:1398] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:47.384076 5082 tablet_bootstrap.cc:492] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:47.384186 5082 ts_tablet_manager.cc:1367] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:47.384641 5082 raft_consensus.cc:348] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:47.384727 5082 raft_consensus.cc:374] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.384747 5082 raft_consensus.cc:729] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.384847 5082 consensus_queue.cc:260] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:47.385123 5082 ts_tablet_manager.cc:1398] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:47.385293 5082 tablet_bootstrap.cc:492] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:47.386193 5082 tablet_bootstrap.cc:654] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.388082 5082 tablet_bootstrap.cc:492] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:47.388177 5082 ts_tablet_manager.cc:1367] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:47.388654 5082 raft_consensus.cc:348] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.388769 5082 raft_consensus.cc:374] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.390565 5082 raft_consensus.cc:729] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.390805 5082 consensus_queue.cc:260] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.391294 5082 ts_tablet_manager.cc:1398] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:00:47.465864 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:47.471587 5077 tablet.cc:1722] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:47.471987 5077 tablet.cc:1620] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:47.473011 5047 tablet.cc:1722] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:47.473183 5079 tablet.cc:1722] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:47.473325 5047 tablet.cc:1620] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:47.473461 5079 tablet.cc:1620] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: MemRowSet was empty: no flush needed.
I20240502 14:00:47.480459 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=2c8a16f241de4db3bb0d105eb61c8446] alter complete (version 2)
I20240502 14:00:47.585486 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:47.592284 5077 tablet.cc:1722] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:47.594054 5079 tablet.cc:1722] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:47.595280 5047 tablet.cc:1722] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:47.601509 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=2c8a16f241de4db3bb0d105eb61c8446] alter complete (version 3)
I20240502 14:00:47.602523 5069 raft_consensus.cc:482] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:47.602622 5069 raft_consensus.cc:504] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:47.603096 5069 leader_election.cc:288] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:47.603701 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "336053fb7de24efd943708fd39dab990" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:47.603868 405 raft_consensus.cc:2454] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:47.604185 2821 leader_election.cc:302] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:47.604183 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "336053fb7de24efd943708fd39dab990" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:47.604305 3283 raft_consensus.cc:2454] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:47.604357 5069 raft_consensus.cc:2781] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:47.604408 5069 raft_consensus.cc:482] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:47.604449 5069 raft_consensus.cc:3037] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.605623 5069 raft_consensus.cc:504] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:47.605993 5069 leader_election.cc:288] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:47.606295 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "336053fb7de24efd943708fd39dab990" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:47.606406 3283 raft_consensus.cc:3037] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.606731 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "336053fb7de24efd943708fd39dab990" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:47.606865 405 raft_consensus.cc:3037] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.607734 5048 raft_consensus.cc:482] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:47.607774 405 raft_consensus.cc:2454] T 336053fb7de24efd943708fd39dab990 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:47.607864 5048 raft_consensus.cc:504] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:47.608215 5048 leader_election.cc:288] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:47.608589 3283 raft_consensus.cc:2454] T 336053fb7de24efd943708fd39dab990 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:47.608665 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49fa0956a8a74ee087bbeb605942e219" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:47.608780 405 raft_consensus.cc:2454] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:47.609090 608 leader_election.cc:302] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:47.609318 5048 raft_consensus.cc:2781] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:47.609366 5048 raft_consensus.cc:482] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:47.609408 5048 raft_consensus.cc:3037] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.610291 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49fa0956a8a74ee087bbeb605942e219" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:47.610422 804 raft_consensus.cc:2454] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:47.610459 5048 raft_consensus.cc:504] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:47.608942 2820 leader_election.cc:302] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:47.610807 5048 leader_election.cc:288] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:47.611075 5069 raft_consensus.cc:2781] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:47.611101 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49fa0956a8a74ee087bbeb605942e219" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:47.611173 5069 raft_consensus.cc:686] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:47.611204 409 raft_consensus.cc:3037] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.611263 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49fa0956a8a74ee087bbeb605942e219" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:47.611371 804 raft_consensus.cc:3037] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.611371 5069 consensus_queue.cc:237] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:47.612270 409 raft_consensus.cc:2454] T 49fa0956a8a74ee087bbeb605942e219 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:47.612355 804 raft_consensus.cc:2454] T 49fa0956a8a74ee087bbeb605942e219 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:47.612600 608 leader_election.cc:302] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:47.612968 5048 raft_consensus.cc:2781] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:47.613076 5048 raft_consensus.cc:686] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:47.613226 5048 consensus_queue.cc:237] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:47.614214 32726 catalog_manager.cc:5484] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:47.616349 32733 catalog_manager.cc:5484] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:47.644807 5095 raft_consensus.cc:482] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:47.644937 5095 raft_consensus.cc:504] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.645366 5095 leader_election.cc:288] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:47.645689 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c47fb9f1470349b3be140e7b70811e7b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:47.645723 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c47fb9f1470349b3be140e7b70811e7b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:47.645839 2884 raft_consensus.cc:2454] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:47.645843 409 raft_consensus.cc:2454] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:47.646135 2953 leader_election.cc:302] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:47.646406 5095 raft_consensus.cc:2781] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:47.646503 5095 raft_consensus.cc:482] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:47.646620 5095 raft_consensus.cc:3037] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.648025 5095 raft_consensus.cc:504] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.648512 5095 leader_election.cc:288] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:47.648684 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c47fb9f1470349b3be140e7b70811e7b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:47.648725 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c47fb9f1470349b3be140e7b70811e7b" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:47.648802 2884 raft_consensus.cc:3037] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.648859 409 raft_consensus.cc:3037] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.649631 2884 raft_consensus.cc:2454] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:47.649744 409 raft_consensus.cc:2454] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:47.649904 2954 leader_election.cc:302] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:47.650058 5095 raft_consensus.cc:2781] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:47.650236 5095 raft_consensus.cc:686] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:47.650372 5095 consensus_queue.cc:237] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.652829 32733 catalog_manager.cc:5484] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:47.706892 32733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:47.707067 32733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:47.709230 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 9435e5c1425d4b308c70f72ce718db30
I20240502 14:00:47.709795 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 9435e5c1425d4b308c70f72ce718db30 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:47 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:47.709944 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 9435e5c1425d4b308c70f72ce718db30 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:47 UTC) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:47.709991 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 9435e5c1425d4b308c70f72ce718db30 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:47 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:47.710115 5101 tablet_replica.cc:330] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:47.710243 5101 raft_consensus.cc:2229] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:47.710319 5103 tablet_replica.cc:330] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:47.710404 5101 raft_consensus.cc:2258] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:47.710434 5102 tablet_replica.cc:330] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:47.710619 5102 raft_consensus.cc:2229] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:47.710494 5103 raft_consensus.cc:2229] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:47.710765 5102 raft_consensus.cc:2258] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:47.710949 5103 raft_consensus.cc:2258] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:47.710987 5101 ts_tablet_manager.cc:1875] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:47.711500 5103 ts_tablet_manager.cc:1875] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:47.711505 5102 ts_tablet_manager.cc:1875] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:47.713111 5101 ts_tablet_manager.cc:1888] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:47.713181 5101 log.cc:1196] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/9435e5c1425d4b308c70f72ce718db30
I20240502 14:00:47.713573 5101 ts_tablet_manager.cc:1909] T 9435e5c1425d4b308c70f72ce718db30 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:47.713747 5102 ts_tablet_manager.cc:1888] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:47.713791 5103 ts_tablet_manager.cc:1888] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:47.713853 5103 log.cc:1196] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/9435e5c1425d4b308c70f72ce718db30
I20240502 14:00:47.713936 5102 log.cc:1196] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/9435e5c1425d4b308c70f72ce718db30
I20240502 14:00:47.714185 5103 ts_tablet_manager.cc:1909] T 9435e5c1425d4b308c70f72ce718db30 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:47.714320 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 9435e5c1425d4b308c70f72ce718db30 (table rebalancer_dynamic_table_new_name [id=2c8a16f241de4db3bb0d105eb61c8446]) successfully deleted
I20240502 14:00:47.714426 5102 ts_tablet_manager.cc:1909] T 9435e5c1425d4b308c70f72ce718db30 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting consensus metadata
I20240502 14:00:47.714962 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 9435e5c1425d4b308c70f72ce718db30 (table rebalancer_dynamic_table_new_name [id=2c8a16f241de4db3bb0d105eb61c8446]) successfully deleted
I20240502 14:00:47.715050 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 9435e5c1425d4b308c70f72ce718db30 (table rebalancer_dynamic_table_new_name [id=2c8a16f241de4db3bb0d105eb61c8446]) successfully deleted
I20240502 14:00:47.811182 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:47.817124 3264 tablet_service.cc:1467] Processing CreateTablet for tablet aba195452d054b4a95b24dca476d5504 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7b9d94c88b534d8087ef8f14dff7ca81]), partition=
I20240502 14:00:47.817224 2999 tablet_service.cc:1467] Processing CreateTablet for tablet aba195452d054b4a95b24dca476d5504 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7b9d94c88b534d8087ef8f14dff7ca81]), partition=
I20240502 14:00:47.817224 520 tablet_service.cc:1467] Processing CreateTablet for tablet aba195452d054b4a95b24dca476d5504 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7b9d94c88b534d8087ef8f14dff7ca81]), partition=
I20240502 14:00:47.817425 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba195452d054b4a95b24dca476d5504. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.817483 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba195452d054b4a95b24dca476d5504. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.817483 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba195452d054b4a95b24dca476d5504. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.819666 5085 tablet_bootstrap.cc:492] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:47.819708 5086 tablet_bootstrap.cc:492] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:47.820581 5085 tablet_bootstrap.cc:654] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.820581 5086 tablet_bootstrap.cc:654] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.820719 5105 tablet_bootstrap.cc:492] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:47.821571 5105 tablet_bootstrap.cc:654] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:47.822238 5085 tablet_bootstrap.cc:492] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:47.822335 5085 ts_tablet_manager.cc:1367] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:47.822820 5085 raft_consensus.cc:348] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.822918 5085 raft_consensus.cc:374] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.822963 5085 raft_consensus.cc:729] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.823088 5085 consensus_queue.cc:260] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.824730 5085 ts_tablet_manager.cc:1398] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:47.825508 5086 tablet_bootstrap.cc:492] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:47.825616 5086 ts_tablet_manager.cc:1367] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:47.826038 5086 raft_consensus.cc:348] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.826138 5086 raft_consensus.cc:374] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.826172 5086 raft_consensus.cc:729] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.826340 5086 consensus_queue.cc:260] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.826628 5086 ts_tablet_manager.cc:1398] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:47.828233 5105 tablet_bootstrap.cc:492] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:47.828347 5105 ts_tablet_manager.cc:1367] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 14:00:47.829042 5105 raft_consensus.cc:348] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.829155 5105 raft_consensus.cc:374] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:47.829186 5105 raft_consensus.cc:729] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:47.829310 5105 consensus_queue.cc:260] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.829622 5105 ts_tablet_manager.cc:1398] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 14:00:47.923632 5104 raft_consensus.cc:482] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:47.923797 5104 raft_consensus.cc:504] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.924235 5104 leader_election.cc:288] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:47.924679 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba195452d054b4a95b24dca476d5504" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:47.924811 3018 raft_consensus.cc:2454] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:47.925113 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba195452d054b4a95b24dca476d5504" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:47.925243 541 raft_consensus.cc:2454] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:47.926069 3216 leader_election.cc:302] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:47.926550 5104 raft_consensus.cc:2781] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:47.926647 5104 raft_consensus.cc:482] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:47.926699 5104 raft_consensus.cc:3037] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.927635 5104 raft_consensus.cc:504] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.927978 5104 leader_election.cc:288] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:47.929172 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba195452d054b4a95b24dca476d5504" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:47.929209 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba195452d054b4a95b24dca476d5504" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:47.929288 541 raft_consensus.cc:3037] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.929328 3018 raft_consensus.cc:3037] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:47.930370 3018 raft_consensus.cc:2454] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:47.930400 541 raft_consensus.cc:2454] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:47.930665 3216 leader_election.cc:302] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:47.930970 5104 raft_consensus.cc:2781] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:47.931061 5104 raft_consensus.cc:686] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:47.931216 5104 consensus_queue.cc:237] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:47.933488 32732 catalog_manager.cc:5484] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:48.045452 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:48.050127 3018 raft_consensus.cc:1261] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:48.050302 541 raft_consensus.cc:1261] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:48.050618 4637 consensus_queue.cc:1035] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.050940 5104 consensus_queue.cc:1035] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.053894 32732 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet aba195452d054b4a95b24dca476d5504 (table rebalancer_dynamic_table [id=7b9d94c88b534d8087ef8f14dff7ca81]). Expected version 1 got 0
I20240502 14:00:48.054867 5114 tablet.cc:1722] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:48.055263 5114 tablet.cc:1620] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:48.056811 32732 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet aba195452d054b4a95b24dca476d5504 (table rebalancer_dynamic_table [id=7b9d94c88b534d8087ef8f14dff7ca81]). Expected version 1 got 0
I20240502 14:00:48.057538 5115 tablet.cc:1722] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:48.058105 5115 tablet.cc:1620] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:48.060283 5047 tablet.cc:1722] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:48.060623 5047 tablet.cc:1620] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:48.061524 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=7b9d94c88b534d8087ef8f14dff7ca81] alter complete (version 1)
I20240502 14:00:48.061949 32732 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet aba195452d054b4a95b24dca476d5504 (table rebalancer_dynamic_table [id=7b9d94c88b534d8087ef8f14dff7ca81]). Expected version 1 got 0
I20240502 14:00:48.073743 5095 consensus_queue.cc:1035] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.077232 5038 consensus_queue.cc:1035] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.118746 4657 consensus_queue.cc:1035] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.121670 5122 consensus_queue.cc:1035] T 336053fb7de24efd943708fd39dab990 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.129218 5048 consensus_queue.cc:1035] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.132622 5098 consensus_queue.cc:1035] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.162415 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:48.164896 32726 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:48.171682 520 tablet_service.cc:1467] Processing CreateTablet for tablet a658bde2687344beabf4d8a0ff8988e2 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=cb41deb677b547cd82087726509f0f21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.172060 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a658bde2687344beabf4d8a0ff8988e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.172163 2999 tablet_service.cc:1467] Processing CreateTablet for tablet a658bde2687344beabf4d8a0ff8988e2 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=cb41deb677b547cd82087726509f0f21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.172426 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a658bde2687344beabf4d8a0ff8988e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.174424 5105 tablet_bootstrap.cc:492] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:48.174675 5085 tablet_bootstrap.cc:492] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:48.175346 5105 tablet_bootstrap.cc:654] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.175710 5085 tablet_bootstrap.cc:654] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.176307 5114 tablet.cc:1722] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:48.176599 5114 tablet.cc:1620] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:48.176801 5047 tablet.cc:1722] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:48.177084 5047 tablet.cc:1620] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:48.177348 522 tablet_service.cc:1467] Processing CreateTablet for tablet 167e1eb3a4034f3ca75e721b8140d757 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=cb41deb677b547cd82087726509f0f21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.177619 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 167e1eb3a4034f3ca75e721b8140d757. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.178073 5085 tablet_bootstrap.cc:492] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:48.178145 5085 ts_tablet_manager.cc:1367] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:48.178579 5085 raft_consensus.cc:348] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:48.178714 5085 raft_consensus.cc:374] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.178751 5085 raft_consensus.cc:729] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.178789 5105 tablet_bootstrap.cc:492] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:48.178870 5105 ts_tablet_manager.cc:1367] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:48.178880 5085 consensus_queue.cc:260] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:48.179157 5085 ts_tablet_manager.cc:1398] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:48.179107 2866 tablet_service.cc:1467] Processing CreateTablet for tablet abe463eb326b4b2681af4d20e5d98698 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=cb41deb677b547cd82087726509f0f21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.179388 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abe463eb326b4b2681af4d20e5d98698. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.179327 5105 raft_consensus.cc:348] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:48.179443 5105 raft_consensus.cc:374] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.179476 5105 raft_consensus.cc:729] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.179582 5105 consensus_queue.cc:260] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:48.179867 5105 ts_tablet_manager.cc:1398] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:48.180017 5105 tablet_bootstrap.cc:492] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:48.180089 3132 tablet_service.cc:1467] Processing CreateTablet for tablet abe463eb326b4b2681af4d20e5d98698 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=cb41deb677b547cd82087726509f0f21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.180349 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abe463eb326b4b2681af4d20e5d98698. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.180925 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 167e1eb3a4034f3ca75e721b8140d757 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=cb41deb677b547cd82087726509f0f21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.181164 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 167e1eb3a4034f3ca75e721b8140d757. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.183454 785 tablet_service.cc:1467] Processing CreateTablet for tablet 167e1eb3a4034f3ca75e721b8140d757 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=cb41deb677b547cd82087726509f0f21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.183737 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 167e1eb3a4034f3ca75e721b8140d757. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.184401 5132 tablet_bootstrap.cc:492] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:48.185104 5115 tablet.cc:1722] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:48.185424 5115 tablet.cc:1620] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:48.185892 5132 tablet_bootstrap.cc:654] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.187269 520 tablet_service.cc:1467] Processing CreateTablet for tablet abe463eb326b4b2681af4d20e5d98698 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=cb41deb677b547cd82087726509f0f21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.187366 653 tablet_service.cc:1467] Processing CreateTablet for tablet a658bde2687344beabf4d8a0ff8988e2 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=cb41deb677b547cd82087726509f0f21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.187547 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abe463eb326b4b2681af4d20e5d98698. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.187677 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a658bde2687344beabf4d8a0ff8988e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.190048 5136 tablet_bootstrap.cc:492] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:48.190932 5105 tablet_bootstrap.cc:654] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.191012 5136 tablet_bootstrap.cc:654] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.192309 5136 tablet_bootstrap.cc:492] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:48.192405 5136 ts_tablet_manager.cc:1367] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:48.192857 5136 raft_consensus.cc:348] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:48.192976 5136 raft_consensus.cc:374] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.193007 5136 raft_consensus.cc:729] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.193120 5136 consensus_queue.cc:260] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:48.193373 5136 ts_tablet_manager.cc:1398] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:48.193701 5133 tablet_bootstrap.cc:492] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:48.195223 32726 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=7b9d94c88b534d8087ef8f14dff7ca81] alter complete (version 2)
I20240502 14:00:48.196678 5134 tablet_bootstrap.cc:492] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:48.198133 5134 tablet_bootstrap.cc:654] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.198292 5133 tablet_bootstrap.cc:654] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.199334 5134 tablet_bootstrap.cc:492] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:48.199429 5134 ts_tablet_manager.cc:1367] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:48.199441 5132 tablet_bootstrap.cc:492] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:48.199553 5132 ts_tablet_manager.cc:1367] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:00:48.199692 5105 tablet_bootstrap.cc:492] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:48.199764 5105 ts_tablet_manager.cc:1367] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.001s
I20240502 14:00:48.199893 5134 raft_consensus.cc:348] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:48.200024 5134 raft_consensus.cc:374] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.200063 5134 raft_consensus.cc:729] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.200066 5132 raft_consensus.cc:348] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:48.200170 5132 raft_consensus.cc:374] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.200202 5132 raft_consensus.cc:729] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.200194 5134 consensus_queue.cc:260] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:48.200248 5105 raft_consensus.cc:348] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:48.200352 5105 raft_consensus.cc:374] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.200331 5132 consensus_queue.cc:260] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:48.200387 5105 raft_consensus.cc:729] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.200450 5134 ts_tablet_manager.cc:1398] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:48.200500 5105 consensus_queue.cc:260] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:48.200636 5132 ts_tablet_manager.cc:1398] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:48.200790 5105 ts_tablet_manager.cc:1398] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:48.200939 5105 tablet_bootstrap.cc:492] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:48.201867 5105 tablet_bootstrap.cc:654] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.202060 5133 tablet_bootstrap.cc:492] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:48.202165 5133 ts_tablet_manager.cc:1367] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:48.202944 5105 tablet_bootstrap.cc:492] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:48.203936 5105 ts_tablet_manager.cc:1367] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:48.204442 5105 raft_consensus.cc:348] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:48.204551 5105 raft_consensus.cc:374] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.204587 5105 raft_consensus.cc:729] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.204715 5105 consensus_queue.cc:260] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:48.204967 5105 ts_tablet_manager.cc:1398] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:48.205176 5133 raft_consensus.cc:348] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:48.205381 5133 raft_consensus.cc:374] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.205474 5133 raft_consensus.cc:729] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.205691 5133 consensus_queue.cc:260] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:48.205947 5133 ts_tablet_manager.cc:1398] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:48.207621 5133 tablet_bootstrap.cc:492] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:48.211640 5133 tablet_bootstrap.cc:654] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.212778 5133 tablet_bootstrap.cc:492] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:48.212869 5133 ts_tablet_manager.cc:1367] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:48.213351 5133 raft_consensus.cc:348] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:48.213482 5133 raft_consensus.cc:374] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.213534 5133 raft_consensus.cc:729] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.213656 5133 consensus_queue.cc:260] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:48.213948 5133 ts_tablet_manager.cc:1398] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:48.220683 5048 raft_consensus.cc:482] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:48.220809 5048 raft_consensus.cc:504] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:48.221268 5048 leader_election.cc:288] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:48.221706 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a658bde2687344beabf4d8a0ff8988e2" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:48.221735 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a658bde2687344beabf4d8a0ff8988e2" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:48.221864 541 raft_consensus.cc:2454] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:48.221863 3018 raft_consensus.cc:2454] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:48.222169 606 leader_election.cc:302] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:48.222337 5048 raft_consensus.cc:2781] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:48.222395 5048 raft_consensus.cc:482] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:48.222442 5048 raft_consensus.cc:3037] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.223546 5048 raft_consensus.cc:504] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:48.223959 5048 leader_election.cc:288] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:48.224357 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a658bde2687344beabf4d8a0ff8988e2" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:48.224360 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a658bde2687344beabf4d8a0ff8988e2" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:48.224485 541 raft_consensus.cc:3037] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.224488 3018 raft_consensus.cc:3037] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.225323 541 raft_consensus.cc:2454] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:48.225512 3018 raft_consensus.cc:2454] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:48.225658 606 leader_election.cc:302] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:48.225981 5048 raft_consensus.cc:2781] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:48.226063 5048 raft_consensus.cc:686] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:48.226167 5048 consensus_queue.cc:237] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:48.228613 32733 catalog_manager.cc:5484] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:48.241961 4879 raft_consensus.cc:482] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:48.242092 4879 raft_consensus.cc:504] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:48.242518 4879 leader_election.cc:288] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:48.246526 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "167e1eb3a4034f3ca75e721b8140d757" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:48.246713 541 raft_consensus.cc:2454] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:48.246642 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "167e1eb3a4034f3ca75e721b8140d757" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:48.246955 3151 raft_consensus.cc:2454] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:48.247392 738 leader_election.cc:302] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:48.248598 4879 raft_consensus.cc:2781] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:48.248672 4879 raft_consensus.cc:482] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:48.248713 4879 raft_consensus.cc:3037] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.249833 4879 raft_consensus.cc:504] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:48.250185 4879 leader_election.cc:288] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:48.250609 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "167e1eb3a4034f3ca75e721b8140d757" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:48.250725 3151 raft_consensus.cc:3037] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.251178 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "167e1eb3a4034f3ca75e721b8140d757" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:48.251314 541 raft_consensus.cc:3037] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.252003 3151 raft_consensus.cc:2454] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:48.252110 541 raft_consensus.cc:2454] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:48.252305 741 leader_election.cc:302] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:48.252485 5128 raft_consensus.cc:2781] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:48.252550 5128 raft_consensus.cc:686] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:48.252712 5128 consensus_queue.cc:237] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:48.254979 32733 catalog_manager.cc:5484] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:48.260931 5118 raft_consensus.cc:482] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:48.261035 5118 raft_consensus.cc:504] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:48.261412 5118 leader_election.cc:288] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:48.261750 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe463eb326b4b2681af4d20e5d98698" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:48.261777 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe463eb326b4b2681af4d20e5d98698" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:48.261871 3151 raft_consensus.cc:2454] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:48.261914 2886 raft_consensus.cc:2454] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 0.
I20240502 14:00:48.262148 477 leader_election.cc:302] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:48.262315 5118 raft_consensus.cc:2781] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:48.262372 5118 raft_consensus.cc:482] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:48.262447 5118 raft_consensus.cc:3037] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.263423 5118 raft_consensus.cc:504] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:48.263733 5118 leader_election.cc:288] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:48.264138 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe463eb326b4b2681af4d20e5d98698" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:48.264257 2886 raft_consensus.cc:3037] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.265198 2886 raft_consensus.cc:2454] T abe463eb326b4b2681af4d20e5d98698 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:48.265493 477 leader_election.cc:302] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:48.265519 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe463eb326b4b2681af4d20e5d98698" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:48.265650 3151 raft_consensus.cc:3037] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.265656 5118 raft_consensus.cc:2781] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:48.265723 5118 raft_consensus.cc:686] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:00:48.265861 5118 consensus_queue.cc:237] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:48.266892 3151 raft_consensus.cc:2454] T abe463eb326b4b2681af4d20e5d98698 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:00:48.268445 32733 catalog_manager.cc:5484] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 0 to 1, leader changed from <none> to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:48.298878 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:48.304490 5114 tablet.cc:1722] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:48.304665 5115 tablet.cc:1722] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:48.305557 5047 tablet.cc:1722] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:48.311236 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=7b9d94c88b534d8087ef8f14dff7ca81] alter complete (version 3)
I20240502 14:00:48.414266 32733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:48.414479 32733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:48.416954 32733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet aba195452d054b4a95b24dca476d5504
I20240502 14:00:48.417745 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet aba195452d054b4a95b24dca476d5504 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:48 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:48.417865 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet aba195452d054b4a95b24dca476d5504 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:48 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:48.417904 520 tablet_service.cc:1514] Processing DeleteTablet for tablet aba195452d054b4a95b24dca476d5504 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:48 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:48.418200 5147 tablet_replica.cc:330] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:48.418226 5148 tablet_replica.cc:330] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:48.418233 5146 tablet_replica.cc:330] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:48.418352 5146 raft_consensus.cc:2229] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:48.418381 5148 raft_consensus.cc:2229] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:48.418434 5147 raft_consensus.cc:2229] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:48.418514 5148 raft_consensus.cc:2258] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:48.418516 5146 raft_consensus.cc:2258] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:48.418715 5147 raft_consensus.cc:2258] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:48.419164 5148 ts_tablet_manager.cc:1875] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:48.419198 5146 ts_tablet_manager.cc:1875] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:48.419267 5147 ts_tablet_manager.cc:1875] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:48.421627 5146 ts_tablet_manager.cc:1888] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:48.421698 5146 log.cc:1196] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/aba195452d054b4a95b24dca476d5504
I20240502 14:00:48.421748 5147 ts_tablet_manager.cc:1888] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:48.421851 5147 log.cc:1196] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/aba195452d054b4a95b24dca476d5504
I20240502 14:00:48.422006 5148 ts_tablet_manager.cc:1888] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:48.422088 5148 log.cc:1196] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/aba195452d054b4a95b24dca476d5504
I20240502 14:00:48.422220 5147 ts_tablet_manager.cc:1909] T aba195452d054b4a95b24dca476d5504 P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:48.422247 5146 ts_tablet_manager.cc:1909] T aba195452d054b4a95b24dca476d5504 P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:48.422434 5148 ts_tablet_manager.cc:1909] T aba195452d054b4a95b24dca476d5504 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:48.422961 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet aba195452d054b4a95b24dca476d5504 (table rebalancer_dynamic_table_new_name [id=7b9d94c88b534d8087ef8f14dff7ca81]) successfully deleted
I20240502 14:00:48.423156 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet aba195452d054b4a95b24dca476d5504 (table rebalancer_dynamic_table_new_name [id=7b9d94c88b534d8087ef8f14dff7ca81]) successfully deleted
I20240502 14:00:48.423286 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet aba195452d054b4a95b24dca476d5504 (table rebalancer_dynamic_table_new_name [id=7b9d94c88b534d8087ef8f14dff7ca81]) successfully deleted
I20240502 14:00:48.518529 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:48.524896 386 tablet_service.cc:1467] Processing CreateTablet for tablet d43aa4a8271843898a269b1f73f7304c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1eae7c108e5e49a3bb93204c04cef120]), partition=
I20240502 14:00:48.525067 520 tablet_service.cc:1467] Processing CreateTablet for tablet d43aa4a8271843898a269b1f73f7304c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1eae7c108e5e49a3bb93204c04cef120]), partition=
I20240502 14:00:48.525089 3264 tablet_service.cc:1467] Processing CreateTablet for tablet d43aa4a8271843898a269b1f73f7304c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1eae7c108e5e49a3bb93204c04cef120]), partition=
I20240502 14:00:48.525233 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43aa4a8271843898a269b1f73f7304c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.525275 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43aa4a8271843898a269b1f73f7304c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.525312 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d43aa4a8271843898a269b1f73f7304c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.527462 5105 tablet_bootstrap.cc:492] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:48.528517 5105 tablet_bootstrap.cc:654] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.528578 5149 tablet_bootstrap.cc:492] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:48.528576 5150 tablet_bootstrap.cc:492] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:48.529528 5149 tablet_bootstrap.cc:654] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.529901 5150 tablet_bootstrap.cc:654] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.530293 5105 tablet_bootstrap.cc:492] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:48.530381 5105 ts_tablet_manager.cc:1367] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:48.530640 5149 tablet_bootstrap.cc:492] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:48.530714 5149 ts_tablet_manager.cc:1367] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:48.530880 5105 raft_consensus.cc:348] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.530999 5105 raft_consensus.cc:374] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.531033 5105 raft_consensus.cc:729] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.531143 5105 consensus_queue.cc:260] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.531136 5149 raft_consensus.cc:348] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.531224 5149 raft_consensus.cc:374] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.531257 5149 raft_consensus.cc:729] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.531363 5149 consensus_queue.cc:260] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.531457 5105 ts_tablet_manager.cc:1398] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:48.531657 5149 ts_tablet_manager.cc:1398] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:48.531955 5150 tablet_bootstrap.cc:492] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:48.532032 5150 ts_tablet_manager.cc:1367] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:48.532440 5150 raft_consensus.cc:348] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.532562 5150 raft_consensus.cc:374] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.532604 5150 raft_consensus.cc:729] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.532732 5150 consensus_queue.cc:260] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.532928 5096 raft_consensus.cc:482] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:48.533035 5150 ts_tablet_manager.cc:1398] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:48.533027 5096 raft_consensus.cc:504] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.533505 5096 leader_election.cc:288] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:48.535914 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43aa4a8271843898a269b1f73f7304c" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:48.536049 541 raft_consensus.cc:2454] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:48.536337 341 leader_election.cc:302] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:48.536549 5096 raft_consensus.cc:2781] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:48.536602 5096 raft_consensus.cc:482] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:48.536656 5096 raft_consensus.cc:3037] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.537667 5096 raft_consensus.cc:504] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.537879 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43aa4a8271843898a269b1f73f7304c" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:48.537994 5096 leader_election.cc:288] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:48.538004 3284 raft_consensus.cc:2454] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:48.538290 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43aa4a8271843898a269b1f73f7304c" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:48.538391 541 raft_consensus.cc:3037] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.539448 541 raft_consensus.cc:2454] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:48.539645 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d43aa4a8271843898a269b1f73f7304c" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:48.539752 3284 raft_consensus.cc:3037] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.539755 341 leader_election.cc:302] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:48.540760 3284 raft_consensus.cc:2454] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:48.541052 5096 raft_consensus.cc:2781] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:48.541141 5096 raft_consensus.cc:686] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:48.541271 5096 consensus_queue.cc:237] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.543982 32733 catalog_manager.cc:5484] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:48.636442 5098 consensus_queue.cc:1035] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.641249 5048 consensus_queue.cc:1035] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.649356 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:48.656203 541 raft_consensus.cc:1261] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:48.657032 5001 consensus_queue.cc:1035] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.658593 3284 raft_consensus.cc:1261] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:48.659015 5001 consensus_queue.cc:1035] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.661826 5047 tablet.cc:1722] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:48.662150 5047 tablet.cc:1620] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:48.663065 32730 catalog_manager.cc:5576] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) does not have the latest schema for tablet d43aa4a8271843898a269b1f73f7304c (table rebalancer_dynamic_table [id=1eae7c108e5e49a3bb93204c04cef120]). Expected version 1 got 0
I20240502 14:00:48.664245 5161 tablet.cc:1722] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:48.664336 32730 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet d43aa4a8271843898a269b1f73f7304c (table rebalancer_dynamic_table [id=1eae7c108e5e49a3bb93204c04cef120]). Expected version 1 got 0
I20240502 14:00:48.664726 32733 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet d43aa4a8271843898a269b1f73f7304c (table rebalancer_dynamic_table [id=1eae7c108e5e49a3bb93204c04cef120]). Expected version 1 got 0
I20240502 14:00:48.664979 5161 tablet.cc:1620] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:48.668074 5114 tablet.cc:1722] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:48.668354 5114 tablet.cc:1620] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:48.671567 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=1eae7c108e5e49a3bb93204c04cef120] alter complete (version 1)
I20240502 14:00:48.777143 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:48.784504 5161 tablet.cc:1722] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:48.784827 5161 tablet.cc:1620] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:48.784962 5047 tablet.cc:1722] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:48.785290 5047 tablet.cc:1620] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:48.784862 32731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:48.791446 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=1eae7c108e5e49a3bb93204c04cef120] alter complete (version 2)
I20240502 14:00:48.792330 5114 tablet.cc:1722] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:48.792704 5114 tablet.cc:1620] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: MemRowSet was empty: no flush needed.
I20240502 14:00:48.798570 5128 consensus_queue.cc:1035] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.806378 3264 tablet_service.cc:1467] Processing CreateTablet for tablet abb4c422dd4240da8bcc83e1b3e32fa4 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=27ea56f6577d467493763287f9354377]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.806380 784 tablet_service.cc:1467] Processing CreateTablet for tablet 59dd5ee0b3554ac493ce2d4d15a99abd (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=27ea56f6577d467493763287f9354377]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.806561 653 tablet_service.cc:1467] Processing CreateTablet for tablet 59dd5ee0b3554ac493ce2d4d15a99abd (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=27ea56f6577d467493763287f9354377]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.806598 785 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab1a0788bee474e8508bf71bb9a6373 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=27ea56f6577d467493763287f9354377]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.806687 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb4c422dd4240da8bcc83e1b3e32fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.806694 784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59dd5ee0b3554ac493ce2d4d15a99abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.806805 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59dd5ee0b3554ac493ce2d4d15a99abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.806816 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ab1a0788bee474e8508bf71bb9a6373. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.808357 520 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab1a0788bee474e8508bf71bb9a6373 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=27ea56f6577d467493763287f9354377]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.808638 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ab1a0788bee474e8508bf71bb9a6373. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.809393 654 tablet_service.cc:1467] Processing CreateTablet for tablet abb4c422dd4240da8bcc83e1b3e32fa4 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=27ea56f6577d467493763287f9354377]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.809639 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb4c422dd4240da8bcc83e1b3e32fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.811333 5149 tablet_bootstrap.cc:492] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:48.811288 2866 tablet_service.cc:1467] Processing CreateTablet for tablet abb4c422dd4240da8bcc83e1b3e32fa4 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=27ea56f6577d467493763287f9354377]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.811564 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb4c422dd4240da8bcc83e1b3e32fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.811954 386 tablet_service.cc:1467] Processing CreateTablet for tablet 59dd5ee0b3554ac493ce2d4d15a99abd (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=27ea56f6577d467493763287f9354377]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.812206 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59dd5ee0b3554ac493ce2d4d15a99abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.812683 5149 tablet_bootstrap.cc:654] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.814064 5105 tablet_bootstrap.cc:492] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:48.814594 5150 tablet_bootstrap.cc:492] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:48.815161 5105 tablet_bootstrap.cc:654] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.815567 5150 tablet_bootstrap.cc:654] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.816043 5167 tablet_bootstrap.cc:492] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:48.816502 5128 consensus_queue.cc:1035] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.819555 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab1a0788bee474e8508bf71bb9a6373 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=27ea56f6577d467493763287f9354377]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:48.819711 5105 tablet_bootstrap.cc:492] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:48.819797 5105 ts_tablet_manager.cc:1367] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 14:00:48.819901 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ab1a0788bee474e8508bf71bb9a6373. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:48.820245 5105 raft_consensus.cc:348] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.820344 5105 raft_consensus.cc:374] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.820374 5105 raft_consensus.cc:729] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.820480 5105 consensus_queue.cc:260] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.820736 5105 ts_tablet_manager.cc:1398] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:48.824321 5149 tablet_bootstrap.cc:492] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:48.824432 5149 ts_tablet_manager.cc:1367] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 14:00:48.825065 5149 raft_consensus.cc:348] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:48.825217 5149 raft_consensus.cc:374] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.825259 5149 raft_consensus.cc:729] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.825429 5149 consensus_queue.cc:260] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:48.825812 5149 ts_tablet_manager.cc:1398] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:48.827775 5118 consensus_queue.cc:1035] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.828044 5015 consensus_queue.cc:1035] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:48.836834 5167 tablet_bootstrap.cc:654] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.838279 5169 tablet_bootstrap.cc:492] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:48.839577 5169 tablet_bootstrap.cc:654] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.839893 5174 tablet_bootstrap.cc:492] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:48.841166 5150 tablet_bootstrap.cc:492] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:48.841270 5150 ts_tablet_manager.cc:1367] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.001s
I20240502 14:00:48.841539 5174 tablet_bootstrap.cc:654] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.841823 5150 raft_consensus.cc:348] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:48.841960 5150 raft_consensus.cc:374] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.842003 5150 raft_consensus.cc:729] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.842134 5150 consensus_queue.cc:260] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:48.842196 5173 tablet_bootstrap.cc:492] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:48.842206 5167 tablet_bootstrap.cc:492] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:48.842288 5167 ts_tablet_manager.cc:1367] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20240502 14:00:48.842446 5150 ts_tablet_manager.cc:1398] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:48.842705 5167 raft_consensus.cc:348] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:48.842818 5167 raft_consensus.cc:374] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.842849 5167 raft_consensus.cc:729] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.843030 5167 consensus_queue.cc:260] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:48.843382 5167 ts_tablet_manager.cc:1398] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:48.844283 5167 tablet_bootstrap.cc:492] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:48.845223 5167 tablet_bootstrap.cc:654] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.846443 5167 tablet_bootstrap.cc:492] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:48.846567 5167 ts_tablet_manager.cc:1367] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:48.847095 5167 raft_consensus.cc:348] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:48.847262 5167 raft_consensus.cc:374] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.847348 5167 raft_consensus.cc:729] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.847512 5167 consensus_queue.cc:260] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:48.847889 5167 ts_tablet_manager.cc:1398] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:48.848323 5173 tablet_bootstrap.cc:654] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.849704 5173 tablet_bootstrap.cc:492] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:48.849789 5173 ts_tablet_manager.cc:1367] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:48.856254 5173 raft_consensus.cc:348] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:48.856489 5173 raft_consensus.cc:374] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.856544 5173 raft_consensus.cc:729] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.856676 5173 consensus_queue.cc:260] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:48.858984 5173 ts_tablet_manager.cc:1398] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.009s user 0.000s sys 0.001s
I20240502 14:00:48.861138 5173 tablet_bootstrap.cc:492] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:48.863320 5173 tablet_bootstrap.cc:654] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:48.864701 5173 tablet_bootstrap.cc:492] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:48.864790 5173 ts_tablet_manager.cc:1367] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:48.864957 5174 tablet_bootstrap.cc:492] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:48.865043 5174 ts_tablet_manager.cc:1367] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240502 14:00:48.865294 5173 raft_consensus.cc:348] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.865398 5173 raft_consensus.cc:374] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.865430 5173 raft_consensus.cc:729] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.865561 5173 consensus_queue.cc:260] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.865554 5174 raft_consensus.cc:348] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.865649 5174 raft_consensus.cc:374] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.865680 5174 raft_consensus.cc:729] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.865828 5174 consensus_queue.cc:260] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.865876 5173 ts_tablet_manager.cc:1398] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:48.868911 5174 ts_tablet_manager.cc:1398] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:48.869998 5169 tablet_bootstrap.cc:492] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:48.870103 5169 ts_tablet_manager.cc:1367] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20240502 14:00:48.870572 5169 raft_consensus.cc:348] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:48.870690 5169 raft_consensus.cc:374] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:48.870746 5169 raft_consensus.cc:729] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:48.870869 5169 consensus_queue.cc:260] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:48.871146 5169 ts_tablet_manager.cc:1398] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:48.893093 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:48.898618 5161 tablet.cc:1722] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:48.899283 5095 raft_consensus.cc:482] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:48.899392 5095 raft_consensus.cc:504] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.899890 5095 leader_election.cc:288] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:48.900041 5047 tablet.cc:1722] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:48.902029 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab1a0788bee474e8508bf71bb9a6373" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:48.902160 541 raft_consensus.cc:2454] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:48.903062 2951 leader_election.cc:302] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:48.904381 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=1eae7c108e5e49a3bb93204c04cef120] alter complete (version 3)
I20240502 14:00:48.904403 5038 raft_consensus.cc:2781] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:48.904466 5038 raft_consensus.cc:482] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:48.904508 5038 raft_consensus.cc:3037] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.904477 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab1a0788bee474e8508bf71bb9a6373" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:48.904603 804 raft_consensus.cc:2454] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:48.905470 5038 raft_consensus.cc:504] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.905815 5038 leader_election.cc:288] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:48.906157 5114 tablet.cc:1722] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:48.906409 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab1a0788bee474e8508bf71bb9a6373" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:48.906527 541 raft_consensus.cc:3037] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.907394 541 raft_consensus.cc:2454] T 6ab1a0788bee474e8508bf71bb9a6373 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:48.907687 2951 leader_election.cc:302] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:48.907877 5038 raft_consensus.cc:2781] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:48.908030 5038 raft_consensus.cc:686] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:48.908167 5038 consensus_queue.cc:237] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:48.908215 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab1a0788bee474e8508bf71bb9a6373" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:48.908322 804 raft_consensus.cc:3037] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:48.909359 804 raft_consensus.cc:2454] T 6ab1a0788bee474e8508bf71bb9a6373 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:48.911340 32731 catalog_manager.cc:5484] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:49.006937 32731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:49.007148 32731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:49.009547 32731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet d43aa4a8271843898a269b1f73f7304c
I20240502 14:00:49.010217 520 tablet_service.cc:1514] Processing DeleteTablet for tablet d43aa4a8271843898a269b1f73f7304c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:49 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:49.010252 386 tablet_service.cc:1514] Processing DeleteTablet for tablet d43aa4a8271843898a269b1f73f7304c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:49 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:49.010710 3264 tablet_service.cc:1514] Processing DeleteTablet for tablet d43aa4a8271843898a269b1f73f7304c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:49 UTC) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:49.010654 5188 tablet_replica.cc:330] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:49.011013 5188 raft_consensus.cc:2229] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:49.011070 5187 tablet_replica.cc:330] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:49.011168 5189 tablet_replica.cc:330] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:49.011215 5187 raft_consensus.cc:2229] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:49.011343 5189 raft_consensus.cc:2229] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:49.011376 5187 raft_consensus.cc:2258] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:49.011440 5188 raft_consensus.cc:2258] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:49.011554 5189 raft_consensus.cc:2258] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:49.012102 5187 ts_tablet_manager.cc:1875] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:49.012298 5189 ts_tablet_manager.cc:1875] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:49.012298 5188 ts_tablet_manager.cc:1875] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:49.014904 5187 ts_tablet_manager.cc:1888] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:49.015051 5187 log.cc:1196] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/d43aa4a8271843898a269b1f73f7304c
I20240502 14:00:49.015260 5189 ts_tablet_manager.cc:1888] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:49.015431 5189 log.cc:1196] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/d43aa4a8271843898a269b1f73f7304c
I20240502 14:00:49.015460 5187 ts_tablet_manager.cc:1909] T d43aa4a8271843898a269b1f73f7304c P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:49.015774 5189 ts_tablet_manager.cc:1909] T d43aa4a8271843898a269b1f73f7304c P da3f8160557e454f8547f45e80c2f013: Deleting consensus metadata
I20240502 14:00:49.016358 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet d43aa4a8271843898a269b1f73f7304c (table rebalancer_dynamic_table_new_name [id=1eae7c108e5e49a3bb93204c04cef120]) successfully deleted
I20240502 14:00:49.016558 5188 ts_tablet_manager.cc:1888] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:49.016558 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet d43aa4a8271843898a269b1f73f7304c (table rebalancer_dynamic_table_new_name [id=1eae7c108e5e49a3bb93204c04cef120]) successfully deleted
I20240502 14:00:49.016963 5188 log.cc:1196] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/d43aa4a8271843898a269b1f73f7304c
I20240502 14:00:49.017302 5188 ts_tablet_manager.cc:1909] T d43aa4a8271843898a269b1f73f7304c P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:49.017977 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet d43aa4a8271843898a269b1f73f7304c (table rebalancer_dynamic_table_new_name [id=1eae7c108e5e49a3bb93204c04cef120]) successfully deleted
I20240502 14:00:49.071004 5048 raft_consensus.cc:482] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:49.071123 5048 raft_consensus.cc:504] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:49.071487 5048 leader_election.cc:288] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:49.071862 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59dd5ee0b3554ac493ce2d4d15a99abd" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:49.071897 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59dd5ee0b3554ac493ce2d4d15a99abd" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:49.072002 409 raft_consensus.cc:2454] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:49.072028 804 raft_consensus.cc:2454] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:49.072290 609 leader_election.cc:302] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:49.072497 5155 raft_consensus.cc:2781] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:49.072556 5155 raft_consensus.cc:482] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:49.072602 5155 raft_consensus.cc:3037] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.073818 5155 raft_consensus.cc:504] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:49.074182 5155 leader_election.cc:288] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:49.074687 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59dd5ee0b3554ac493ce2d4d15a99abd" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:49.074808 804 raft_consensus.cc:3037] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.075664 804 raft_consensus.cc:2454] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:49.075971 609 leader_election.cc:302] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:49.076174 5155 raft_consensus.cc:2781] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:49.076156 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59dd5ee0b3554ac493ce2d4d15a99abd" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:49.076277 409 raft_consensus.cc:3037] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.076267 5155 raft_consensus.cc:686] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:49.076417 5155 consensus_queue.cc:237] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:49.077369 409 raft_consensus.cc:2454] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:49.079041 32731 catalog_manager.cc:5484] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:49.102615 5048 raft_consensus.cc:482] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:49.102747 5048 raft_consensus.cc:504] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:49.103263 5048 leader_election.cc:288] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:49.103632 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb4c422dd4240da8bcc83e1b3e32fa4" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:49.103639 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb4c422dd4240da8bcc83e1b3e32fa4" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:49.103781 3284 raft_consensus.cc:2454] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:49.103780 2886 raft_consensus.cc:2454] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:49.104125 609 leader_election.cc:302] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:49.104488 5048 raft_consensus.cc:2781] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:49.104544 5048 raft_consensus.cc:482] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:49.104588 5048 raft_consensus.cc:3037] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.105412 5048 raft_consensus.cc:504] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:49.106073 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb4c422dd4240da8bcc83e1b3e32fa4" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:49.106186 3284 raft_consensus.cc:3037] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.106582 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb4c422dd4240da8bcc83e1b3e32fa4" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:49.106702 2886 raft_consensus.cc:3037] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.107220 3284 raft_consensus.cc:2454] T abb4c422dd4240da8bcc83e1b3e32fa4 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:49.107513 2886 raft_consensus.cc:2454] T abb4c422dd4240da8bcc83e1b3e32fa4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:49.107748 607 leader_election.cc:302] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:49.107913 5048 leader_election.cc:288] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:49.107944 5155 raft_consensus.cc:2781] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:49.108029 5155 raft_consensus.cc:686] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:49.108172 5155 consensus_queue.cc:237] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:49.110680 32731 catalog_manager.cc:5484] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:49.111944 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:49.117146 520 tablet_service.cc:1467] Processing CreateTablet for tablet f3207e82f58c4bdea60c179413a7eb4b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c9879b65ebfe4d759d2fe80d1c3a55b8]), partition=
I20240502 14:00:49.117460 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3207e82f58c4bdea60c179413a7eb4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.117640 3130 tablet_service.cc:1467] Processing CreateTablet for tablet f3207e82f58c4bdea60c179413a7eb4b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c9879b65ebfe4d759d2fe80d1c3a55b8]), partition=
I20240502 14:00:49.117875 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3207e82f58c4bdea60c179413a7eb4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.119726 5105 tablet_bootstrap.cc:492] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:49.120229 5194 tablet_bootstrap.cc:492] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:49.120380 2999 tablet_service.cc:1467] Processing CreateTablet for tablet f3207e82f58c4bdea60c179413a7eb4b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c9879b65ebfe4d759d2fe80d1c3a55b8]), partition=
I20240502 14:00:49.120610 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3207e82f58c4bdea60c179413a7eb4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.120668 5105 tablet_bootstrap.cc:654] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.121153 5194 tablet_bootstrap.cc:654] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.122301 5105 tablet_bootstrap.cc:492] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:49.122387 5105 ts_tablet_manager.cc:1367] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:49.122866 5105 raft_consensus.cc:348] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:49.122982 5105 raft_consensus.cc:374] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.123019 5105 raft_consensus.cc:729] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.123153 5105 consensus_queue.cc:260] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:49.123456 5105 ts_tablet_manager.cc:1398] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:49.124085 5194 tablet_bootstrap.cc:492] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:49.124181 5194 ts_tablet_manager.cc:1367] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:49.124677 5194 raft_consensus.cc:348] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:49.124855 5174 tablet_bootstrap.cc:492] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:49.124960 5194 raft_consensus.cc:374] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.125010 5194 raft_consensus.cc:729] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.125154 5194 consensus_queue.cc:260] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:49.125488 5194 ts_tablet_manager.cc:1398] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:49.125710 5174 tablet_bootstrap.cc:654] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.126757 5174 tablet_bootstrap.cc:492] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:49.126817 5174 ts_tablet_manager.cc:1367] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:49.127257 5174 raft_consensus.cc:348] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:49.127367 5174 raft_consensus.cc:374] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.127396 5174 raft_consensus.cc:729] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.127516 5174 consensus_queue.cc:260] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:49.127775 5174 ts_tablet_manager.cc:1398] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:49.214044 4972 raft_consensus.cc:482] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:49.214161 4972 raft_consensus.cc:504] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:49.214614 4972 leader_election.cc:288] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:49.214967 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3207e82f58c4bdea60c179413a7eb4b" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:49.215101 541 raft_consensus.cc:2454] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:49.215323 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3207e82f58c4bdea60c179413a7eb4b" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:49.215452 3019 raft_consensus.cc:2454] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:49.215747 3084 leader_election.cc:302] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:49.215967 4972 raft_consensus.cc:2781] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:49.216033 4972 raft_consensus.cc:482] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:49.216070 4972 raft_consensus.cc:3037] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.217207 4972 raft_consensus.cc:504] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:49.217540 4972 leader_election.cc:288] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:49.217847 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3207e82f58c4bdea60c179413a7eb4b" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:49.217978 541 raft_consensus.cc:3037] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.217911 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3207e82f58c4bdea60c179413a7eb4b" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:49.218061 3019 raft_consensus.cc:3037] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.218708 541 raft_consensus.cc:2454] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:49.219002 3084 leader_election.cc:302] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:49.219033 3019 raft_consensus.cc:2454] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:49.219187 4972 raft_consensus.cc:2781] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:49.219322 4972 raft_consensus.cc:686] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:49.219462 4972 consensus_queue.cc:237] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:49.221725 32731 catalog_manager.cc:5484] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:49.346491 32731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:49.347087 5185 consensus_queue.cc:1035] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:49.351354 3019 raft_consensus.cc:1261] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:49.351960 540 raft_consensus.cc:1261] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:49.352028 5078 consensus_queue.cc:1035] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:49.352721 5095 consensus_queue.cc:1035] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:49.354035 5078 consensus_queue.cc:1035] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:49.359685 32731 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet f3207e82f58c4bdea60c179413a7eb4b (table rebalancer_dynamic_table [id=c9879b65ebfe4d759d2fe80d1c3a55b8]). Expected version 1 got 0
I20240502 14:00:49.361291 5207 tablet.cc:1722] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:49.361840 5207 tablet.cc:1620] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:49.362649 32730 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet f3207e82f58c4bdea60c179413a7eb4b (table rebalancer_dynamic_table [id=c9879b65ebfe4d759d2fe80d1c3a55b8]). Expected version 1 got 0
I20240502 14:00:49.363269 32731 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet f3207e82f58c4bdea60c179413a7eb4b (table rebalancer_dynamic_table [id=c9879b65ebfe4d759d2fe80d1c3a55b8]). Expected version 1 got 0
I20240502 14:00:49.364888 32730 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet f3207e82f58c4bdea60c179413a7eb4b (table rebalancer_dynamic_table [id=c9879b65ebfe4d759d2fe80d1c3a55b8]). Expected version 1 got 0
I20240502 14:00:49.366529 5047 tablet.cc:1722] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:49.366847 5047 tablet.cc:1620] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:49.367925 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=c9879b65ebfe4d759d2fe80d1c3a55b8] alter complete (version 1)
I20240502 14:00:49.368062 5201 mvcc.cc:205] Tried to move back new op lower bound from 7023241008539598848 to 7023241008004169728. Current Snapshot: MvccSnapshot[applied={T|T < 7023241008539598848}]
I20240502 14:00:49.368400 5208 tablet.cc:1722] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:49.368815 5208 tablet.cc:1620] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:49.370471 32731 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet f3207e82f58c4bdea60c179413a7eb4b (table rebalancer_dynamic_table [id=c9879b65ebfe4d759d2fe80d1c3a55b8]). Expected version 1 got 0
I20240502 14:00:49.469671 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:49.475004 5207 tablet.cc:1722] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:49.475366 5207 tablet.cc:1620] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:49.475627 5208 tablet.cc:1722] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:49.476045 5208 tablet.cc:1620] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:49.476366 5047 tablet.cc:1722] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:49.476662 5047 tablet.cc:1620] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:49.483340 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=c9879b65ebfe4d759d2fe80d1c3a55b8] alter complete (version 2)
I20240502 14:00:49.543354 5098 consensus_queue.cc:1035] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:49.547387 5048 consensus_queue.cc:1035] T abb4c422dd4240da8bcc83e1b3e32fa4 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:49.572211 5098 consensus_queue.cc:1035] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:49.575290 5098 consensus_queue.cc:1035] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:49.586158 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:49.593021 5207 tablet.cc:1722] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:49.593976 5047 tablet.cc:1722] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:49.595753 5208 tablet.cc:1722] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:49.600818 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=c9879b65ebfe4d759d2fe80d1c3a55b8] alter complete (version 3)
I20240502 14:00:49.617142 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:49.623992 3130 tablet_service.cc:1467] Processing CreateTablet for tablet f68734d21e1843d9ad58832164dfcaae (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=17276d22bba34d4ca997f1b6b58889c2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:49.624137 3264 tablet_service.cc:1467] Processing CreateTablet for tablet f68734d21e1843d9ad58832164dfcaae (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=17276d22bba34d4ca997f1b6b58889c2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:49.624351 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f68734d21e1843d9ad58832164dfcaae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.624433 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f68734d21e1843d9ad58832164dfcaae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.624796 386 tablet_service.cc:1467] Processing CreateTablet for tablet b8afd1dff8f040eb802bc7b1a4b4a26a (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=17276d22bba34d4ca997f1b6b58889c2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:49.625044 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8afd1dff8f040eb802bc7b1a4b4a26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.626209 5217 tablet_bootstrap.cc:492] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:49.626705 2999 tablet_service.cc:1467] Processing CreateTablet for tablet b8afd1dff8f040eb802bc7b1a4b4a26a (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=17276d22bba34d4ca997f1b6b58889c2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:49.626960 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8afd1dff8f040eb802bc7b1a4b4a26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.627285 654 tablet_service.cc:1467] Processing CreateTablet for tablet de7ca321195f4bfebefa2df5ecda109c (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=17276d22bba34d4ca997f1b6b58889c2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:49.627378 5218 tablet_bootstrap.cc:492] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:49.627552 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de7ca321195f4bfebefa2df5ecda109c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.627595 520 tablet_service.cc:1467] Processing CreateTablet for tablet f68734d21e1843d9ad58832164dfcaae (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=17276d22bba34d4ca997f1b6b58889c2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:49.627960 5194 tablet_bootstrap.cc:492] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:49.628662 5218 tablet_bootstrap.cc:654] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.628918 5194 tablet_bootstrap.cc:654] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.629545 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f68734d21e1843d9ad58832164dfcaae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.630008 5174 tablet_bootstrap.cc:492] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:49.630393 5217 tablet_bootstrap.cc:654] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.630753 5221 tablet_bootstrap.cc:492] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:49.630926 5174 tablet_bootstrap.cc:654] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.631438 5218 tablet_bootstrap.cc:492] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:49.631521 5218 ts_tablet_manager.cc:1367] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:49.632009 5224 tablet_bootstrap.cc:492] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:49.632089 5174 tablet_bootstrap.cc:492] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:49.632056 5218 raft_consensus.cc:348] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.632158 5218 raft_consensus.cc:374] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.632180 5174 ts_tablet_manager.cc:1367] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:49.632189 5218 raft_consensus.cc:729] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.632680 3264 tablet_service.cc:1467] Processing CreateTablet for tablet de7ca321195f4bfebefa2df5ecda109c (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=17276d22bba34d4ca997f1b6b58889c2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:49.632763 5174 raft_consensus.cc:348] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.632866 5174 raft_consensus.cc:374] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.632926 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de7ca321195f4bfebefa2df5ecda109c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.632900 5174 raft_consensus.cc:729] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.632972 5224 tablet_bootstrap.cc:654] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.633036 5174 consensus_queue.cc:260] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.633335 5174 ts_tablet_manager.cc:1398] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:49.634327 5194 tablet_bootstrap.cc:492] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:49.634415 5194 ts_tablet_manager.cc:1367] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:49.634783 785 tablet_service.cc:1467] Processing CreateTablet for tablet b8afd1dff8f040eb802bc7b1a4b4a26a (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=17276d22bba34d4ca997f1b6b58889c2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:49.634878 5194 raft_consensus.cc:348] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:49.634970 5194 raft_consensus.cc:374] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.635010 5194 raft_consensus.cc:729] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.635077 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8afd1dff8f040eb802bc7b1a4b4a26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.635126 5194 consensus_queue.cc:260] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:49.635365 5218 consensus_queue.cc:260] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.635430 5194 ts_tablet_manager.cc:1398] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:49.635946 2866 tablet_service.cc:1467] Processing CreateTablet for tablet de7ca321195f4bfebefa2df5ecda109c (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=17276d22bba34d4ca997f1b6b58889c2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:49.636231 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de7ca321195f4bfebefa2df5ecda109c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.636659 5218 ts_tablet_manager.cc:1398] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:49.639686 5217 tablet_bootstrap.cc:492] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:49.639554 5227 tablet_bootstrap.cc:492] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:49.639792 5217 ts_tablet_manager.cc:1367] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 14:00:49.640867 5217 raft_consensus.cc:348] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:49.640972 5217 raft_consensus.cc:374] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.641005 5217 raft_consensus.cc:729] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.641127 5217 consensus_queue.cc:260] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:49.641428 5217 ts_tablet_manager.cc:1398] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:49.641628 5217 tablet_bootstrap.cc:492] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:49.642463 5217 tablet_bootstrap.cc:654] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.642915 5227 tablet_bootstrap.cc:654] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.643862 5221 tablet_bootstrap.cc:654] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.643635 5217 tablet_bootstrap.cc:492] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:49.644253 5217 ts_tablet_manager.cc:1367] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:49.644587 5226 tablet_bootstrap.cc:492] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:49.644721 5217 raft_consensus.cc:348] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:49.644878 5217 raft_consensus.cc:374] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.644956 5221 tablet_bootstrap.cc:492] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:49.645004 5217 raft_consensus.cc:729] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.645180 5217 consensus_queue.cc:260] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:49.645223 5221 ts_tablet_manager.cc:1367] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.001s
I20240502 14:00:49.645610 5217 ts_tablet_manager.cc:1398] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:49.645761 5227 tablet_bootstrap.cc:492] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:49.645849 5227 ts_tablet_manager.cc:1367] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 14:00:49.645906 5221 raft_consensus.cc:348] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:49.646020 5221 raft_consensus.cc:374] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.646051 5221 raft_consensus.cc:729] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.646195 5221 consensus_queue.cc:260] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:49.646613 5221 ts_tablet_manager.cc:1398] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:49.646926 5226 tablet_bootstrap.cc:654] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.648517 5227 raft_consensus.cc:348] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:49.648850 5227 raft_consensus.cc:374] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.648954 5227 raft_consensus.cc:729] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.649003 5224 tablet_bootstrap.cc:492] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:49.649103 5224 ts_tablet_manager.cc:1367] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 14:00:49.649194 5227 consensus_queue.cc:260] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:49.649600 5224 raft_consensus.cc:348] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:49.649713 5224 raft_consensus.cc:374] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.649729 5226 tablet_bootstrap.cc:492] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:49.649749 5224 raft_consensus.cc:729] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.649801 5226 ts_tablet_manager.cc:1367] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:49.649881 5224 consensus_queue.cc:260] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:49.650143 5224 ts_tablet_manager.cc:1398] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:49.650360 5227 ts_tablet_manager.cc:1398] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 14:00:49.654000 5226 raft_consensus.cc:348] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.654112 5226 raft_consensus.cc:374] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.654142 5226 raft_consensus.cc:729] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.654255 5226 consensus_queue.cc:260] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.658057 5226 ts_tablet_manager.cc:1398] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240502 14:00:49.695056 5104 raft_consensus.cc:482] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:49.695171 5104 raft_consensus.cc:504] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:49.695585 5104 leader_election.cc:288] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:49.695915 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de7ca321195f4bfebefa2df5ecda109c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:49.695991 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de7ca321195f4bfebefa2df5ecda109c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:49.696065 2884 raft_consensus.cc:2454] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:49.696120 672 raft_consensus.cc:2454] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:49.696429 3216 leader_election.cc:302] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:49.696611 5104 raft_consensus.cc:2781] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:49.696684 5104 raft_consensus.cc:482] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:49.696738 5104 raft_consensus.cc:3037] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.697705 5104 raft_consensus.cc:504] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:49.698014 5104 leader_election.cc:288] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:49.698307 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de7ca321195f4bfebefa2df5ecda109c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:49.698315 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de7ca321195f4bfebefa2df5ecda109c" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:49.698421 672 raft_consensus.cc:3037] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.698434 2884 raft_consensus.cc:3037] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.699208 2884 raft_consensus.cc:2454] T de7ca321195f4bfebefa2df5ecda109c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:49.699215 672 raft_consensus.cc:2454] T de7ca321195f4bfebefa2df5ecda109c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:49.699484 3219 leader_election.cc:302] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:49.699621 5104 raft_consensus.cc:2781] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:49.699708 5104 raft_consensus.cc:686] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:49.699864 5104 consensus_queue.cc:237] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:00:49.701944 32733 catalog_manager.cc:5484] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:49.702379 5001 raft_consensus.cc:482] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:49.702504 5001 raft_consensus.cc:504] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.703032 5001 leader_election.cc:288] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:49.703305 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8afd1dff8f040eb802bc7b1a4b4a26a" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:49.703438 3018 raft_consensus.cc:2454] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:49.703743 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8afd1dff8f040eb802bc7b1a4b4a26a" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:49.703917 805 raft_consensus.cc:2454] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:49.703763 32732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:49.704028 32732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:49.704041 341 leader_election.cc:302] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:49.704209 5001 raft_consensus.cc:2781] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:49.704274 5001 raft_consensus.cc:482] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:49.704331 5001 raft_consensus.cc:3037] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.705281 5001 raft_consensus.cc:504] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.705611 5001 leader_election.cc:288] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:49.705987 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8afd1dff8f040eb802bc7b1a4b4a26a" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:49.706108 805 raft_consensus.cc:3037] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.706117 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8afd1dff8f040eb802bc7b1a4b4a26a" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:49.706239 3018 raft_consensus.cc:3037] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.706796 32732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet f3207e82f58c4bdea60c179413a7eb4b
I20240502 14:00:49.707137 805 raft_consensus.cc:2454] T b8afd1dff8f040eb802bc7b1a4b4a26a P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:49.707244 3018 raft_consensus.cc:2454] T b8afd1dff8f040eb802bc7b1a4b4a26a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:49.707504 344 leader_election.cc:302] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:49.707633 520 tablet_service.cc:1514] Processing DeleteTablet for tablet f3207e82f58c4bdea60c179413a7eb4b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:49 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:49.707660 5001 raft_consensus.cc:2781] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:49.707747 5001 raft_consensus.cc:686] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:49.707901 5001 consensus_queue.cc:237] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.708338 3130 tablet_service.cc:1514] Processing DeleteTablet for tablet f3207e82f58c4bdea60c179413a7eb4b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:49 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:49.708570 5233 tablet_replica.cc:330] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:49.708727 5233 raft_consensus.cc:2229] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:49.708981 5233 raft_consensus.cc:2258] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:49.709098 5235 tablet_replica.cc:330] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:49.709314 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet f3207e82f58c4bdea60c179413a7eb4b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:49 UTC) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:49.709506 5235 raft_consensus.cc:2229] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:49.709782 5235 raft_consensus.cc:2258] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:49.709976 5233 ts_tablet_manager.cc:1875] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:49.710505 5235 ts_tablet_manager.cc:1875] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:49.710423 32732 catalog_manager.cc:5484] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:49.713022 5233 ts_tablet_manager.cc:1888] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:49.713095 5233 log.cc:1196] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/f3207e82f58c4bdea60c179413a7eb4b
I20240502 14:00:49.713409 5233 ts_tablet_manager.cc:1909] T f3207e82f58c4bdea60c179413a7eb4b P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:49.713845 5236 tablet_replica.cc:330] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:49.714007 5236 raft_consensus.cc:2229] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:49.714146 5236 raft_consensus.cc:2258] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:49.714792 5235 ts_tablet_manager.cc:1888] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:49.714751 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet f3207e82f58c4bdea60c179413a7eb4b (table rebalancer_dynamic_table_new_name [id=c9879b65ebfe4d759d2fe80d1c3a55b8]) successfully deleted
I20240502 14:00:49.714867 5235 log.cc:1196] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/f3207e82f58c4bdea60c179413a7eb4b
I20240502 14:00:49.715027 5236 ts_tablet_manager.cc:1875] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:49.715217 5235 ts_tablet_manager.cc:1909] T f3207e82f58c4bdea60c179413a7eb4b P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:49.715811 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet f3207e82f58c4bdea60c179413a7eb4b (table rebalancer_dynamic_table_new_name [id=c9879b65ebfe4d759d2fe80d1c3a55b8]) successfully deleted
I20240502 14:00:49.716888 5236 ts_tablet_manager.cc:1888] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:49.717015 5236 log.cc:1196] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/f3207e82f58c4bdea60c179413a7eb4b
I20240502 14:00:49.717298 5236 ts_tablet_manager.cc:1909] T f3207e82f58c4bdea60c179413a7eb4b P 452ecbc085e54db49c3b01c20a96f860: Deleting consensus metadata
I20240502 14:00:49.717962 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet f3207e82f58c4bdea60c179413a7eb4b (table rebalancer_dynamic_table_new_name [id=c9879b65ebfe4d759d2fe80d1c3a55b8]) successfully deleted
I20240502 14:00:49.798354 5104 raft_consensus.cc:482] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:49.798461 5104 raft_consensus.cc:504] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:49.798826 5104 leader_election.cc:288] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:49.799194 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68734d21e1843d9ad58832164dfcaae" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:49.799211 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68734d21e1843d9ad58832164dfcaae" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:49.799335 3151 raft_consensus.cc:2454] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:49.799345 540 raft_consensus.cc:2454] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:49.799574 3216 leader_election.cc:302] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:49.799744 5104 raft_consensus.cc:2781] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:49.799858 5104 raft_consensus.cc:482] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:49.799921 5104 raft_consensus.cc:3037] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.800794 5104 raft_consensus.cc:504] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:49.801188 5104 leader_election.cc:288] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:49.801466 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68734d21e1843d9ad58832164dfcaae" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:49.801585 3151 raft_consensus.cc:3037] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.801682 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f68734d21e1843d9ad58832164dfcaae" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:49.801808 541 raft_consensus.cc:3037] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:49.802624 3151 raft_consensus.cc:2454] T f68734d21e1843d9ad58832164dfcaae P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:49.802770 541 raft_consensus.cc:2454] T f68734d21e1843d9ad58832164dfcaae P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:49.803025 3219 leader_election.cc:302] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:49.803375 4994 raft_consensus.cc:2781] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:49.803452 4994 raft_consensus.cc:686] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:49.803576 4994 consensus_queue.cc:237] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:49.806414 32732 catalog_manager.cc:5484] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:49.809790 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:49.815011 520 tablet_service.cc:1467] Processing CreateTablet for tablet 92b36646f435435f89e8f74c7e95313d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a498d970d2624b8e9a9dcc68bf867029]), partition=
I20240502 14:00:49.815140 386 tablet_service.cc:1467] Processing CreateTablet for tablet 92b36646f435435f89e8f74c7e95313d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a498d970d2624b8e9a9dcc68bf867029]), partition=
I20240502 14:00:49.815320 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b36646f435435f89e8f74c7e95313d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.815371 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b36646f435435f89e8f74c7e95313d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.817541 5218 tablet_bootstrap.cc:492] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:49.817541 5224 tablet_bootstrap.cc:492] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:49.817731 785 tablet_service.cc:1467] Processing CreateTablet for tablet 92b36646f435435f89e8f74c7e95313d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a498d970d2624b8e9a9dcc68bf867029]), partition=
I20240502 14:00:49.817974 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b36646f435435f89e8f74c7e95313d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:49.818542 5224 tablet_bootstrap.cc:654] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.818542 5218 tablet_bootstrap.cc:654] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.819710 5226 tablet_bootstrap.cc:492] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:49.819756 5224 tablet_bootstrap.cc:492] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:49.819860 5224 ts_tablet_manager.cc:1367] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:49.820291 5224 raft_consensus.cc:348] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.820415 5224 raft_consensus.cc:374] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.820457 5224 raft_consensus.cc:729] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.820582 5224 consensus_queue.cc:260] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.820693 5226 tablet_bootstrap.cc:654] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:49.820873 5224 ts_tablet_manager.cc:1398] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:49.821573 5218 tablet_bootstrap.cc:492] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:49.821653 5218 ts_tablet_manager.cc:1367] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:49.821859 5226 tablet_bootstrap.cc:492] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:49.821952 5226 ts_tablet_manager.cc:1367] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:49.822072 5218 raft_consensus.cc:348] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.822196 5218 raft_consensus.cc:374] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.822243 5218 raft_consensus.cc:729] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.822383 5218 consensus_queue.cc:260] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.822373 5226 raft_consensus.cc:348] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.822479 5226 raft_consensus.cc:374] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:49.822520 5226 raft_consensus.cc:729] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:49.822628 5226 consensus_queue.cc:260] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:49.822924 5226 ts_tablet_manager.cc:1398] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:49.822932 5218 ts_tablet_manager.cc:1398] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:50.102474 5104 consensus_queue.cc:1035] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.106985 4994 consensus_queue.cc:1035] T de7ca321195f4bfebefa2df5ecda109c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.152065 5199 raft_consensus.cc:482] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:50.152199 5199 raft_consensus.cc:504] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:50.152601 5199 leader_election.cc:288] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:50.152897 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b36646f435435f89e8f74c7e95313d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:50.152932 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b36646f435435f89e8f74c7e95313d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:50.153059 540 raft_consensus.cc:2454] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:50.153064 409 raft_consensus.cc:2454] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:50.153318 738 leader_election.cc:302] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:50.153570 5199 raft_consensus.cc:2781] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:50.153631 5199 raft_consensus.cc:482] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:50.153682 5199 raft_consensus.cc:3037] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.154837 5199 raft_consensus.cc:504] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:50.155193 5199 leader_election.cc:288] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:50.155501 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b36646f435435f89e8f74c7e95313d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:50.155509 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b36646f435435f89e8f74c7e95313d" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:50.155639 409 raft_consensus.cc:3037] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.155647 540 raft_consensus.cc:3037] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.156679 540 raft_consensus.cc:2454] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:50.156704 409 raft_consensus.cc:2454] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:50.156962 738 leader_election.cc:302] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:50.157126 5199 raft_consensus.cc:2781] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:50.157218 5199 raft_consensus.cc:686] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:50.157352 5199 consensus_queue.cc:237] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:00:50.159960 32730 catalog_manager.cc:5484] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:50.182489 5104 consensus_queue.cc:1035] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.185931 5202 consensus_queue.cc:1035] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.192203 5096 consensus_queue.cc:1035] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.195005 5096 consensus_queue.cc:1035] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.286981 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:50.290916 405 raft_consensus.cc:1261] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:50.290946 540 raft_consensus.cc:1261] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:50.291383 5128 consensus_queue.cc:1035] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.291529 5199 consensus_queue.cc:1035] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.294852 32733 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 92b36646f435435f89e8f74c7e95313d (table rebalancer_dynamic_table [id=a498d970d2624b8e9a9dcc68bf867029]). Expected version 1 got 0
I20240502 14:00:50.296689 5264 tablet.cc:1722] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:50.297041 32733 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet 92b36646f435435f89e8f74c7e95313d (table rebalancer_dynamic_table [id=a498d970d2624b8e9a9dcc68bf867029]). Expected version 1 got 0
I20240502 14:00:50.297632 5264 tablet.cc:1620] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:50.297644 32732 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 92b36646f435435f89e8f74c7e95313d (table rebalancer_dynamic_table [id=a498d970d2624b8e9a9dcc68bf867029]). Expected version 1 got 0
I20240502 14:00:50.297981 32730 catalog_manager.cc:5576] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715) does not have the latest schema for tablet 92b36646f435435f89e8f74c7e95313d (table rebalancer_dynamic_table [id=a498d970d2624b8e9a9dcc68bf867029]). Expected version 1 got 0
I20240502 14:00:50.298611 5260 tablet.cc:1722] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:50.298871 5261 tablet.cc:1722] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:50.298946 5260 tablet.cc:1620] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:50.299232 5261 tablet.cc:1620] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:50.300346 32730 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet 92b36646f435435f89e8f74c7e95313d (table rebalancer_dynamic_table [id=a498d970d2624b8e9a9dcc68bf867029]). Expected version 1 got 0
I20240502 14:00:50.303169 32731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=a498d970d2624b8e9a9dcc68bf867029] alter complete (version 1)
I20240502 14:00:50.335026 32732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:50.341809 386 tablet_service.cc:1467] Processing CreateTablet for tablet 8a81f3de2e784508b730efbbd2d78127 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=c4ee0314ba8244b5aa832bb31e2b0534]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:50.342092 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 10d6cc99575f4ebb8c980e9cb3e35145 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=c4ee0314ba8244b5aa832bb31e2b0534]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:50.342186 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a81f3de2e784508b730efbbd2d78127. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.342160 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 10d6cc99575f4ebb8c980e9cb3e35145 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=c4ee0314ba8244b5aa832bb31e2b0534]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:50.342353 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10d6cc99575f4ebb8c980e9cb3e35145. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.342487 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10d6cc99575f4ebb8c980e9cb3e35145. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.342645 3264 tablet_service.cc:1467] Processing CreateTablet for tablet 8a81f3de2e784508b730efbbd2d78127 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=c4ee0314ba8244b5aa832bb31e2b0534]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:50.342905 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a81f3de2e784508b730efbbd2d78127. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.344128 654 tablet_service.cc:1467] Processing CreateTablet for tablet 8a81f3de2e784508b730efbbd2d78127 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=c4ee0314ba8244b5aa832bb31e2b0534]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:50.344406 654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a81f3de2e784508b730efbbd2d78127. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.345307 2866 tablet_service.cc:1467] Processing CreateTablet for tablet 72590739196d48ac88d69b5bcd1124b0 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=c4ee0314ba8244b5aa832bb31e2b0534]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:50.345551 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72590739196d48ac88d69b5bcd1124b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.345638 653 tablet_service.cc:1467] Processing CreateTablet for tablet 72590739196d48ac88d69b5bcd1124b0 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=c4ee0314ba8244b5aa832bb31e2b0534]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:50.345866 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72590739196d48ac88d69b5bcd1124b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.346758 5268 tablet_bootstrap.cc:492] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:50.346921 520 tablet_service.cc:1467] Processing CreateTablet for tablet 10d6cc99575f4ebb8c980e9cb3e35145 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=c4ee0314ba8244b5aa832bb31e2b0534]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:50.347226 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10d6cc99575f4ebb8c980e9cb3e35145. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.347967 5268 tablet_bootstrap.cc:654] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.348254 5265 tablet_bootstrap.cc:492] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:50.348765 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 72590739196d48ac88d69b5bcd1124b0 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=c4ee0314ba8244b5aa832bb31e2b0534]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:50.349009 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72590739196d48ac88d69b5bcd1124b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.349328 5265 tablet_bootstrap.cc:654] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.350185 5270 tablet_bootstrap.cc:492] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:50.351919 5270 tablet_bootstrap.cc:654] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.352293 5266 tablet_bootstrap.cc:492] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:50.353224 5266 tablet_bootstrap.cc:654] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.354399 5266 tablet_bootstrap.cc:492] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:50.354614 5265 tablet_bootstrap.cc:492] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:50.354638 5266 ts_tablet_manager.cc:1367] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:50.354712 5268 tablet_bootstrap.cc:492] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:50.354871 5265 ts_tablet_manager.cc:1367] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:50.354869 5268 ts_tablet_manager.cc:1367] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240502 14:00:50.355111 5266 raft_consensus.cc:348] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:50.355296 5266 raft_consensus.cc:374] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.355383 5266 raft_consensus.cc:729] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.355518 5265 raft_consensus.cc:348] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:50.355592 5266 consensus_queue.cc:260] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:50.355777 5265 raft_consensus.cc:374] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.355919 5265 raft_consensus.cc:729] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.356102 5265 consensus_queue.cc:260] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:50.356593 5265 ts_tablet_manager.cc:1398] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:50.356803 5265 tablet_bootstrap.cc:492] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:50.357077 5266 ts_tablet_manager.cc:1398] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:50.357280 5266 tablet_bootstrap.cc:492] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:50.357543 5268 raft_consensus.cc:348] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:50.357664 5268 raft_consensus.cc:374] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.357702 5268 raft_consensus.cc:729] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.357800 5265 tablet_bootstrap.cc:654] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.357827 5268 consensus_queue.cc:260] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:50.358130 5268 ts_tablet_manager.cc:1398] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20240502 14:00:50.358355 5266 tablet_bootstrap.cc:654] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.359179 5268 tablet_bootstrap.cc:492] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:50.359727 5274 tablet_bootstrap.cc:492] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:50.361056 5274 tablet_bootstrap.cc:654] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.362263 5265 tablet_bootstrap.cc:492] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:50.362401 5265 ts_tablet_manager.cc:1367] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:50.362540 5266 tablet_bootstrap.cc:492] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:50.362691 5266 ts_tablet_manager.cc:1367] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:50.362879 5265 raft_consensus.cc:348] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:50.363032 5265 raft_consensus.cc:374] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.363090 5265 raft_consensus.cc:729] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.363241 5265 consensus_queue.cc:260] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:50.363566 5265 ts_tablet_manager.cc:1398] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:50.356587 5267 tablet_bootstrap.cc:492] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:50.364977 5267 tablet_bootstrap.cc:654] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.363524 5266 raft_consensus.cc:348] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:50.366448 5266 raft_consensus.cc:374] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.366539 5266 raft_consensus.cc:729] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.367056 5266 consensus_queue.cc:260] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:50.367302 5268 tablet_bootstrap.cc:654] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.367781 5266 ts_tablet_manager.cc:1398] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:50.369235 5270 tablet_bootstrap.cc:492] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:50.369355 5270 ts_tablet_manager.cc:1367] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.002s
I20240502 14:00:50.369863 5270 raft_consensus.cc:348] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:50.369976 5270 raft_consensus.cc:374] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.370009 5270 raft_consensus.cc:729] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.370158 5270 consensus_queue.cc:260] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:50.370468 5270 ts_tablet_manager.cc:1398] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:50.371635 5267 tablet_bootstrap.cc:492] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:50.371932 5267 ts_tablet_manager.cc:1367] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:00:50.372332 5274 tablet_bootstrap.cc:492] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:50.372440 5274 ts_tablet_manager.cc:1367] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 14:00:50.372550 5267 raft_consensus.cc:348] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:50.372700 5267 raft_consensus.cc:374] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.372735 5267 raft_consensus.cc:729] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.372862 5267 consensus_queue.cc:260] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:50.372951 5274 raft_consensus.cc:348] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:50.373149 5267 ts_tablet_manager.cc:1398] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:50.373980 5268 tablet_bootstrap.cc:492] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:50.374121 5268 ts_tablet_manager.cc:1367] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20240502 14:00:50.373045 5274 raft_consensus.cc:374] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.374388 5274 raft_consensus.cc:729] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.374564 5274 consensus_queue.cc:260] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:50.374692 5268 raft_consensus.cc:348] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:50.374830 5268 raft_consensus.cc:374] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.374892 5268 raft_consensus.cc:729] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.375088 5268 consensus_queue.cc:260] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:50.375464 5268 ts_tablet_manager.cc:1398] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:50.375941 5274 ts_tablet_manager.cc:1398] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:50.408648 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:50.409061 5096 raft_consensus.cc:482] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:50.409173 5096 raft_consensus.cc:504] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:50.409621 5096 leader_election.cc:288] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:50.409924 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a81f3de2e784508b730efbbd2d78127" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:50.409978 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a81f3de2e784508b730efbbd2d78127" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:50.410079 3283 raft_consensus.cc:2454] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:50.410100 672 raft_consensus.cc:2454] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:50.410444 342 leader_election.cc:302] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:50.411310 5096 raft_consensus.cc:2781] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:50.411372 5096 raft_consensus.cc:482] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:50.411414 5096 raft_consensus.cc:3037] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.412329 5096 raft_consensus.cc:504] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:50.413658 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a81f3de2e784508b730efbbd2d78127" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:50.413796 3283 raft_consensus.cc:3037] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.414770 5096 leader_election.cc:288] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:50.414934 3283 raft_consensus.cc:2454] T 8a81f3de2e784508b730efbbd2d78127 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:50.415251 342 leader_election.cc:302] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:50.415244 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a81f3de2e784508b730efbbd2d78127" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:50.415390 672 raft_consensus.cc:3037] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.415792 5096 raft_consensus.cc:2781] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:50.415918 5260 tablet.cc:1722] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:50.416162 5096 raft_consensus.cc:686] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:50.416262 5260 tablet.cc:1620] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:50.416419 5096 consensus_queue.cc:237] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } }
I20240502 14:00:50.416484 672 raft_consensus.cc:2454] T 8a81f3de2e784508b730efbbd2d78127 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:50.416675 5264 tablet.cc:1722] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:50.416966 5264 tablet.cc:1620] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: MemRowSet was empty: no flush needed.
I20240502 14:00:50.420761 5261 tablet.cc:1722] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:50.421072 5261 tablet.cc:1620] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:50.421556 32730 catalog_manager.cc:5484] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:50.422704 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=a498d970d2624b8e9a9dcc68bf867029] alter complete (version 2)
I20240502 14:00:50.447703 5069 raft_consensus.cc:482] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:50.447873 5069 raft_consensus.cc:504] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:50.448366 5069 leader_election.cc:288] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:50.448721 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10d6cc99575f4ebb8c980e9cb3e35145" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:50.448776 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10d6cc99575f4ebb8c980e9cb3e35145" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:50.448871 3019 raft_consensus.cc:2454] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:50.448900 541 raft_consensus.cc:2454] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 0.
I20240502 14:00:50.449198 2819 leader_election.cc:302] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:50.449398 5069 raft_consensus.cc:2781] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:50.449465 5069 raft_consensus.cc:482] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:50.449512 5069 raft_consensus.cc:3037] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.450624 5069 raft_consensus.cc:504] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:50.450922 5069 leader_election.cc:288] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:50.451231 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10d6cc99575f4ebb8c980e9cb3e35145" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:50.451356 3019 raft_consensus.cc:3037] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.451344 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10d6cc99575f4ebb8c980e9cb3e35145" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:50.451476 541 raft_consensus.cc:3037] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.452323 3019 raft_consensus.cc:2454] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:50.452605 2819 leader_election.cc:302] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 4bcd7712aa844d98977a7d7e605c31f5; no voters:
I20240502 14:00:50.452646 541 raft_consensus.cc:2454] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
I20240502 14:00:50.452788 5069 raft_consensus.cc:2781] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:50.452925 5069 raft_consensus.cc:686] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:50.453081 5069 consensus_queue.cc:237] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:50.455427 32730 catalog_manager.cc:5484] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:50.524796 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:50.530743 5260 tablet.cc:1722] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:50.531366 5264 tablet.cc:1722] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:50.531733 5261 tablet.cc:1722] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:50.539033 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=a498d970d2624b8e9a9dcc68bf867029] alter complete (version 3)
I20240502 14:00:50.630887 5155 raft_consensus.cc:482] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:50.631007 5155 raft_consensus.cc:504] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:50.631392 5155 leader_election.cc:288] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:50.631744 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72590739196d48ac88d69b5bcd1124b0" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:50.631778 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72590739196d48ac88d69b5bcd1124b0" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:50.631913 2886 raft_consensus.cc:2454] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:50.631968 3018 raft_consensus.cc:2454] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 0.
I20240502 14:00:50.632223 609 leader_election.cc:302] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:50.632450 5155 raft_consensus.cc:2781] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:50.632615 5155 raft_consensus.cc:482] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:50.632673 5155 raft_consensus.cc:3037] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.633688 5155 raft_consensus.cc:504] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:50.634025 5155 leader_election.cc:288] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:50.634349 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72590739196d48ac88d69b5bcd1124b0" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:50.634402 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72590739196d48ac88d69b5bcd1124b0" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:50.634505 3018 raft_consensus.cc:3037] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.634516 2886 raft_consensus.cc:3037] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.635537 2886 raft_consensus.cc:2454] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:50.635537 3018 raft_consensus.cc:2454] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:00:50.635890 609 leader_election.cc:302] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9; no voters:
I20240502 14:00:50.636103 5155 raft_consensus.cc:2781] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:50.636211 5155 raft_consensus.cc:686] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:50.636359 5155 consensus_queue.cc:237] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:50.638826 32730 catalog_manager.cc:5484] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:50.641726 32730 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:50.641856 32730 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:50.643627 32730 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 92b36646f435435f89e8f74c7e95313d
I20240502 14:00:50.644215 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 92b36646f435435f89e8f74c7e95313d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:50 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:50.644544 386 tablet_service.cc:1514] Processing DeleteTablet for tablet 92b36646f435435f89e8f74c7e95313d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:50 UTC) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:50.644608 5285 tablet_replica.cc:330] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:50.644614 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 92b36646f435435f89e8f74c7e95313d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:50 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:50.644768 5285 raft_consensus.cc:2229] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:50.644901 5285 raft_consensus.cc:2258] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:50.644963 5287 tablet_replica.cc:330] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:50.645069 5286 tablet_replica.cc:330] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:50.645156 5287 raft_consensus.cc:2229] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:50.645228 5286 raft_consensus.cc:2229] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:50.645413 5286 raft_consensus.cc:2258] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:50.645502 5287 raft_consensus.cc:2258] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:50.645973 5285 ts_tablet_manager.cc:1875] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:50.645998 5286 ts_tablet_manager.cc:1875] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:50.646100 5287 ts_tablet_manager.cc:1875] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:50.648507 5287 ts_tablet_manager.cc:1888] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:50.648519 5286 ts_tablet_manager.cc:1888] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:50.648582 5287 log.cc:1196] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/92b36646f435435f89e8f74c7e95313d
I20240502 14:00:50.648587 5286 log.cc:1196] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/92b36646f435435f89e8f74c7e95313d
I20240502 14:00:50.648519 5285 ts_tablet_manager.cc:1888] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:50.648965 5287 ts_tablet_manager.cc:1909] T 92b36646f435435f89e8f74c7e95313d P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:50.649011 5286 ts_tablet_manager.cc:1909] T 92b36646f435435f89e8f74c7e95313d P f003b732dd0f46958262b42f8d20305e: Deleting consensus metadata
I20240502 14:00:50.649083 5285 log.cc:1196] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/92b36646f435435f89e8f74c7e95313d
I20240502 14:00:50.649770 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 92b36646f435435f89e8f74c7e95313d (table rebalancer_dynamic_table_new_name [id=a498d970d2624b8e9a9dcc68bf867029]) successfully deleted
I20240502 14:00:50.649785 5285 ts_tablet_manager.cc:1909] T 92b36646f435435f89e8f74c7e95313d P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:50.650835 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 92b36646f435435f89e8f74c7e95313d (table rebalancer_dynamic_table_new_name [id=a498d970d2624b8e9a9dcc68bf867029]) successfully deleted
I20240502 14:00:50.651008 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 92b36646f435435f89e8f74c7e95313d (table rebalancer_dynamic_table_new_name [id=a498d970d2624b8e9a9dcc68bf867029]) successfully deleted
I20240502 14:00:50.745540 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:50.752108 520 tablet_service.cc:1467] Processing CreateTablet for tablet 9d9dcb9c047f45d8a958fe3c8a966303 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=90d74dc6b5b343e3af842d5833d5436f]), partition=
I20240502 14:00:50.752167 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 9d9dcb9c047f45d8a958fe3c8a966303 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=90d74dc6b5b343e3af842d5833d5436f]), partition=
I20240502 14:00:50.752420 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d9dcb9c047f45d8a958fe3c8a966303. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.752424 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d9dcb9c047f45d8a958fe3c8a966303. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.754514 5270 tablet_bootstrap.cc:492] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:50.754602 5288 tablet_bootstrap.cc:492] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:50.754680 785 tablet_service.cc:1467] Processing CreateTablet for tablet 9d9dcb9c047f45d8a958fe3c8a966303 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=90d74dc6b5b343e3af842d5833d5436f]), partition=
I20240502 14:00:50.755002 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d9dcb9c047f45d8a958fe3c8a966303. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:50.755789 5288 tablet_bootstrap.cc:654] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.755800 5270 tablet_bootstrap.cc:654] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.757031 5288 tablet_bootstrap.cc:492] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:50.757114 5288 ts_tablet_manager.cc:1367] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:50.757383 5291 tablet_bootstrap.cc:492] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:50.757550 5288 raft_consensus.cc:348] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:50.757650 5288 raft_consensus.cc:374] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.757687 5288 raft_consensus.cc:729] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.757839 5288 consensus_queue.cc:260] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:50.758170 5288 ts_tablet_manager.cc:1398] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:50.758500 5291 tablet_bootstrap.cc:654] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:50.759470 5270 tablet_bootstrap.cc:492] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:50.759563 5270 ts_tablet_manager.cc:1367] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 14:00:50.759996 5270 raft_consensus.cc:348] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:50.760082 5270 raft_consensus.cc:374] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.760102 5270 raft_consensus.cc:729] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.760208 5270 consensus_queue.cc:260] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:50.765892 5270 ts_tablet_manager.cc:1398] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:00:50.767551 5291 tablet_bootstrap.cc:492] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:50.767671 5291 ts_tablet_manager.cc:1367] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:50.768292 5291 raft_consensus.cc:348] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:50.768494 4972 raft_consensus.cc:482] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:50.768523 5291 raft_consensus.cc:374] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:50.768602 4972 raft_consensus.cc:504] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:50.768743 5291 raft_consensus.cc:729] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:50.768950 5291 consensus_queue.cc:260] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:50.769014 4972 leader_election.cc:288] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:50.769326 5291 ts_tablet_manager.cc:1398] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:50.769335 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d9dcb9c047f45d8a958fe3c8a966303" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01" is_pre_election: true
I20240502 14:00:50.769677 805 raft_consensus.cc:2454] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:50.770099 3087 leader_election.cc:302] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:50.770268 4972 raft_consensus.cc:2781] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:50.770265 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d9dcb9c047f45d8a958fe3c8a966303" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:50.770320 4972 raft_consensus.cc:482] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:50.770359 4972 raft_consensus.cc:3037] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.770371 542 raft_consensus.cc:2454] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:50.771579 4972 raft_consensus.cc:504] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:50.771998 4972 leader_election.cc:288] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:50.772457 805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d9dcb9c047f45d8a958fe3c8a966303" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:50.772480 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d9dcb9c047f45d8a958fe3c8a966303" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:50.772562 805 raft_consensus.cc:3037] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.772603 542 raft_consensus.cc:3037] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:50.773628 542 raft_consensus.cc:2454] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:50.773643 805 raft_consensus.cc:2454] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:50.774418 3087 leader_election.cc:302] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:50.774606 4972 raft_consensus.cc:2781] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:50.774698 4972 raft_consensus.cc:686] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:50.774866 4972 consensus_queue.cc:237] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } }
I20240502 14:00:50.777220 32730 catalog_manager.cc:5484] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:50.883335 32730 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:50.887725 542 raft_consensus.cc:1261] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:50.887876 805 raft_consensus.cc:1261] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:50.888178 5252 consensus_queue.cc:1035] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.888351 4972 consensus_queue.cc:1035] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.892014 32730 catalog_manager.cc:5576] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251) does not have the latest schema for tablet 9d9dcb9c047f45d8a958fe3c8a966303 (table rebalancer_dynamic_table [id=90d74dc6b5b343e3af842d5833d5436f]). Expected version 1 got 0
I20240502 14:00:50.892388 32732 catalog_manager.cc:5576] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443) does not have the latest schema for tablet 9d9dcb9c047f45d8a958fe3c8a966303 (table rebalancer_dynamic_table [id=90d74dc6b5b343e3af842d5833d5436f]). Expected version 1 got 0
I20240502 14:00:50.893424 5257 mvcc.cc:205] Tried to move back new op lower bound from 7023241014832295936 to 7023241014375579648. Current Snapshot: MvccSnapshot[applied={T|T < 7023241014832295936}]
I20240502 14:00:50.893662 5260 tablet.cc:1722] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:50.893692 5261 tablet.cc:1722] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:50.893950 5260 tablet.cc:1620] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:50.894013 5261 tablet.cc:1620] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:50.894781 5300 tablet.cc:1722] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:50.895083 5300 tablet.cc:1620] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:50.896620 32732 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 9d9dcb9c047f45d8a958fe3c8a966303 (table rebalancer_dynamic_table [id=90d74dc6b5b343e3af842d5833d5436f]). Expected version 1 got 0
I20240502 14:00:50.899863 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=90d74dc6b5b343e3af842d5833d5436f] alter complete (version 1)
I20240502 14:00:50.933038 4657 consensus_queue.cc:1035] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.937036 5302 consensus_queue.cc:1035] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.960247 5283 consensus_queue.cc:1035] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:50.963372 5001 consensus_queue.cc:1035] T 8a81f3de2e784508b730efbbd2d78127 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:51.003885 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:51.009292 5300 tablet.cc:1722] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:51.009632 5300 tablet.cc:1620] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:51.009934 5260 tablet.cc:1722] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:51.010196 5260 tablet.cc:1620] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:51.016192 5261 tablet.cc:1722] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:51.016541 5261 tablet.cc:1620] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: MemRowSet was empty: no flush needed.
I20240502 14:00:51.017046 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=90d74dc6b5b343e3af842d5833d5436f] alter complete (version 2)
I20240502 14:00:51.069756 5293 consensus_queue.cc:1035] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:51.073912 5293 consensus_queue.cc:1035] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:51.121925 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 14:00:51.127677 5300 tablet.cc:1722] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:51.128494 5260 tablet.cc:1722] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:51.128607 5261 tablet.cc:1722] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 14:00:51.134114 32732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table_new_name [id=90d74dc6b5b343e3af842d5833d5436f] alter complete (version 3)
I20240502 14:00:51.146081 32733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:51.153920 3264 tablet_service.cc:1467] Processing CreateTablet for tablet e5ad9695e69d49b68898b1b1b30e969b (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=59ef62e3607c418eb9ab69360784630f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.154143 386 tablet_service.cc:1467] Processing CreateTablet for tablet e5ad9695e69d49b68898b1b1b30e969b (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=59ef62e3607c418eb9ab69360784630f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.154268 3264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ad9695e69d49b68898b1b1b30e969b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.154477 386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ad9695e69d49b68898b1b1b30e969b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.154902 388 tablet_service.cc:1467] Processing CreateTablet for tablet 81cd2e05295641678fa8d7711c797113 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=59ef62e3607c418eb9ab69360784630f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.154953 520 tablet_service.cc:1467] Processing CreateTablet for tablet 9bf2e05fba1943edb5cd00989f93df7c (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=59ef62e3607c418eb9ab69360784630f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.155115 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81cd2e05295641678fa8d7711c797113. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.155223 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bf2e05fba1943edb5cd00989f93df7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.156114 387 tablet_service.cc:1467] Processing CreateTablet for tablet 9bf2e05fba1943edb5cd00989f93df7c (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=59ef62e3607c418eb9ab69360784630f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.156323 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bf2e05fba1943edb5cd00989f93df7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.157421 5312 tablet_bootstrap.cc:492] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:51.157634 5270 tablet_bootstrap.cc:492] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:51.158902 5270 tablet_bootstrap.cc:654] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.159001 3130 tablet_service.cc:1467] Processing CreateTablet for tablet 9bf2e05fba1943edb5cd00989f93df7c (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=59ef62e3607c418eb9ab69360784630f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.159019 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 81cd2e05295641678fa8d7711c797113 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=59ef62e3607c418eb9ab69360784630f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.159193 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 81cd2e05295641678fa8d7711c797113 (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=59ef62e3607c418eb9ab69360784630f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.159307 3130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bf2e05fba1943edb5cd00989f93df7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.159421 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81cd2e05295641678fa8d7711c797113. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.159618 5312 tablet_bootstrap.cc:654] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.159732 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81cd2e05295641678fa8d7711c797113. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.161463 5288 tablet_bootstrap.cc:492] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:51.162451 5288 tablet_bootstrap.cc:654] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.162700 5315 tablet_bootstrap.cc:492] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:51.163326 5270 tablet_bootstrap.cc:492] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:51.163404 5270 ts_tablet_manager.cc:1367] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:51.163646 2866 tablet_service.cc:1467] Processing CreateTablet for tablet e5ad9695e69d49b68898b1b1b30e969b (DEFAULT_TABLE table=rebalancer_extra_table_74 [id=59ef62e3607c418eb9ab69360784630f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.163936 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ad9695e69d49b68898b1b1b30e969b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.163981 5270 raft_consensus.cc:348] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:51.164098 5315 tablet_bootstrap.cc:654] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.164110 5270 raft_consensus.cc:374] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.164146 5270 raft_consensus.cc:729] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.164283 5270 consensus_queue.cc:260] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:51.164598 5270 ts_tablet_manager.cc:1398] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:51.165402 5315 tablet_bootstrap.cc:492] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:51.165475 5315 ts_tablet_manager.cc:1367] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:51.166383 5318 tablet_bootstrap.cc:492] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:51.166437 5315 raft_consensus.cc:348] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:51.166592 5315 raft_consensus.cc:374] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.166668 5315 raft_consensus.cc:729] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.166951 5315 consensus_queue.cc:260] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:51.167372 5315 ts_tablet_manager.cc:1398] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:51.167604 5315 tablet_bootstrap.cc:492] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:51.168083 5318 tablet_bootstrap.cc:654] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.169234 5288 tablet_bootstrap.cc:492] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:51.169353 5288 ts_tablet_manager.cc:1367] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:51.169859 5288 raft_consensus.cc:348] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:51.169957 5288 raft_consensus.cc:374] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.169994 5288 raft_consensus.cc:729] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.170125 5288 consensus_queue.cc:260] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:51.170908 5288 ts_tablet_manager.cc:1398] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:51.171265 5288 tablet_bootstrap.cc:492] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:51.171293 5315 tablet_bootstrap.cc:654] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.172219 5288 tablet_bootstrap.cc:654] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.172271 5315 tablet_bootstrap.cc:492] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:51.172398 5315 ts_tablet_manager.cc:1367] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:51.171090 5312 tablet_bootstrap.cc:492] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:51.172842 5315 raft_consensus.cc:348] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:51.172883 5312 ts_tablet_manager.cc:1367] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:00:51.172948 5315 raft_consensus.cc:374] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.172991 5315 raft_consensus.cc:729] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.173090 5315 consensus_queue.cc:260] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:51.173349 5315 ts_tablet_manager.cc:1398] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:51.173585 5288 tablet_bootstrap.cc:492] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:51.173677 5288 ts_tablet_manager.cc:1367] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:51.174103 5288 raft_consensus.cc:348] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:51.174221 5288 raft_consensus.cc:374] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.174253 5288 raft_consensus.cc:729] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.174364 5288 consensus_queue.cc:260] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:51.174625 5288 ts_tablet_manager.cc:1398] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:51.174624 5318 tablet_bootstrap.cc:492] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:51.174721 5318 ts_tablet_manager.cc:1367] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240502 14:00:51.173810 5312 raft_consensus.cc:348] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:51.175745 5312 raft_consensus.cc:374] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.175848 5312 raft_consensus.cc:729] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.176025 5312 consensus_queue.cc:260] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:51.177191 5318 raft_consensus.cc:348] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:51.177338 5318 raft_consensus.cc:374] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.177397 5318 raft_consensus.cc:729] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.177556 5318 consensus_queue.cc:260] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:51.177903 5318 ts_tablet_manager.cc:1398] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:00:51.181840 5312 ts_tablet_manager.cc:1398] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20240502 14:00:51.182705 5312 tablet_bootstrap.cc:492] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:51.184832 5312 tablet_bootstrap.cc:654] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.185833 5312 tablet_bootstrap.cc:492] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:51.185909 5312 ts_tablet_manager.cc:1367] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:51.186269 5312 raft_consensus.cc:348] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:51.186372 5312 raft_consensus.cc:374] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.186414 5312 raft_consensus.cc:729] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.186539 5312 consensus_queue.cc:260] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:51.186825 5312 ts_tablet_manager.cc:1398] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:51.187047 5312 tablet_bootstrap.cc:492] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:51.188012 5312 tablet_bootstrap.cc:654] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.191135 5312 tablet_bootstrap.cc:492] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:51.191232 5312 ts_tablet_manager.cc:1367] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:51.191602 5312 raft_consensus.cc:348] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:51.191699 5312 raft_consensus.cc:374] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.191735 5312 raft_consensus.cc:729] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.191893 5312 consensus_queue.cc:260] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:51.192191 5312 ts_tablet_manager.cc:1398] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:51.218119 5299 raft_consensus.cc:482] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:51.218246 5299 raft_consensus.cc:504] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:51.218627 5299 leader_election.cc:288] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:51.218815 5104 raft_consensus.cc:482] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:51.218936 5104 raft_consensus.cc:504] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:51.218991 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf2e05fba1943edb5cd00989f93df7c" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:51.219132 409 raft_consensus.cc:2454] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:51.219131 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf2e05fba1943edb5cd00989f93df7c" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:51.219246 542 raft_consensus.cc:2454] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:51.219386 5104 leader_election.cc:288] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:51.219475 3086 leader_election.cc:302] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:51.219638 5299 raft_consensus.cc:2781] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:51.219707 5299 raft_consensus.cc:482] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:51.219750 5299 raft_consensus.cc:3037] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.220041 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cd2e05295641678fa8d7711c797113" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:51.220157 3151 raft_consensus.cc:2454] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:51.220428 3219 leader_election.cc:302] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:51.220609 5104 raft_consensus.cc:2781] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:51.220669 5104 raft_consensus.cc:482] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:51.220713 5104 raft_consensus.cc:3037] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.220930 5299 raft_consensus.cc:504] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:51.221231 5299 leader_election.cc:288] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:51.221407 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cd2e05295641678fa8d7711c797113" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:51.221524 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf2e05fba1943edb5cd00989f93df7c" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:51.221544 409 raft_consensus.cc:2454] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:51.221601 405 raft_consensus.cc:3037] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.221716 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf2e05fba1943edb5cd00989f93df7c" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:51.221844 542 raft_consensus.cc:3037] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.221913 5104 raft_consensus.cc:504] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:51.222237 5104 leader_election.cc:288] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:51.222510 405 raft_consensus.cc:2454] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:51.222576 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cd2e05295641678fa8d7711c797113" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:51.222666 409 raft_consensus.cc:3037] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.222841 542 raft_consensus.cc:2454] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:51.223107 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cd2e05295641678fa8d7711c797113" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:51.223148 3086 leader_election.cc:302] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:51.223219 3151 raft_consensus.cc:3037] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.223287 5299 raft_consensus.cc:2781] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:51.223456 5299 raft_consensus.cc:686] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:51.223613 5299 consensus_queue.cc:237] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } }
I20240502 14:00:51.223675 409 raft_consensus.cc:2454] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:51.224013 3218 leader_election.cc:302] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:51.224210 3151 raft_consensus.cc:2454] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:51.224792 5104 raft_consensus.cc:2781] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:51.224884 5104 raft_consensus.cc:686] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:51.225008 5104 consensus_queue.cc:237] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } }
I20240502 14:00:51.226652 32733 catalog_manager.cc:5484] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:51.227524 32730 catalog_manager.cc:5484] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:51.237700 32730 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:51.237874 32730 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 14:00:51.240374 32730 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: Sending DeleteTablet for 3 replicas of tablet 9d9dcb9c047f45d8a958fe3c8a966303
I20240502 14:00:51.241577 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d9dcb9c047f45d8a958fe3c8a966303 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:51 UTC) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:51.241983 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d9dcb9c047f45d8a958fe3c8a966303 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:51 UTC) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:51.242280 5327 tablet_replica.cc:330] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:51.242359 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d9dcb9c047f45d8a958fe3c8a966303 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 14:00:51 UTC) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:51.242424 5327 raft_consensus.cc:2229] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:51.242574 5327 raft_consensus.cc:2258] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:51.242739 5328 tablet_replica.cc:330] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:51.242928 5328 raft_consensus.cc:2229] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Raft consensus shutting down.
I20240502 14:00:51.243135 5328 raft_consensus.cc:2258] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:51.243942 5328 ts_tablet_manager.cc:1875] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:51.244702 5327 ts_tablet_manager.cc:1875] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:51.247349 5327 ts_tablet_manager.cc:1888] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:51.247489 5327 log.cc:1196] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/9d9dcb9c047f45d8a958fe3c8a966303
I20240502 14:00:51.247870 5327 ts_tablet_manager.cc:1909] T 9d9dcb9c047f45d8a958fe3c8a966303 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting consensus metadata
I20240502 14:00:51.248632 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 9d9dcb9c047f45d8a958fe3c8a966303 (table rebalancer_dynamic_table_new_name [id=90d74dc6b5b343e3af842d5833d5436f]) successfully deleted
I20240502 14:00:51.248692 5328 ts_tablet_manager.cc:1888] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:51.248762 5328 log.cc:1196] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/9d9dcb9c047f45d8a958fe3c8a966303
I20240502 14:00:51.249056 5328 ts_tablet_manager.cc:1909] T 9d9dcb9c047f45d8a958fe3c8a966303 P 2b6796cf0e2e496e9dd641beb432f485: Deleting consensus metadata
I20240502 14:00:51.249687 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 9d9dcb9c047f45d8a958fe3c8a966303 (table rebalancer_dynamic_table_new_name [id=90d74dc6b5b343e3af842d5833d5436f]) successfully deleted
I20240502 14:00:51.252089 5326 tablet_replica.cc:330] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:51.252254 5326 raft_consensus.cc:2229] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:51.252382 5326 raft_consensus.cc:2258] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:51.253124 5326 ts_tablet_manager.cc:1875] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 14:00:51.255607 5326 ts_tablet_manager.cc:1888] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 14:00:51.255692 5326 log.cc:1196] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/9d9dcb9c047f45d8a958fe3c8a966303
I20240502 14:00:51.256089 5326 ts_tablet_manager.cc:1909] T 9d9dcb9c047f45d8a958fe3c8a966303 P c351a89a5fe34e788b9d8064fbffa751: Deleting consensus metadata
I20240502 14:00:51.256726 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 9d9dcb9c047f45d8a958fe3c8a966303 (table rebalancer_dynamic_table_new_name [id=90d74dc6b5b343e3af842d5833d5436f]) successfully deleted
I20240502 14:00:51.346163 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 14:00:51.352480 3132 tablet_service.cc:1467] Processing CreateTablet for tablet 1be73835d3d04112a4b34ea0a16c91d2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d1f75889e95049d3aaf74b5f91a96411]), partition=
I20240502 14:00:51.352684 785 tablet_service.cc:1467] Processing CreateTablet for tablet 1be73835d3d04112a4b34ea0a16c91d2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d1f75889e95049d3aaf74b5f91a96411]), partition=
I20240502 14:00:51.352790 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1be73835d3d04112a4b34ea0a16c91d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.353026 785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1be73835d3d04112a4b34ea0a16c91d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.355273 5288 tablet_bootstrap.cc:492] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:51.355413 5329 tablet_bootstrap.cc:492] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:51.355546 2999 tablet_service.cc:1467] Processing CreateTablet for tablet 1be73835d3d04112a4b34ea0a16c91d2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d1f75889e95049d3aaf74b5f91a96411]), partition=
I20240502 14:00:51.355849 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1be73835d3d04112a4b34ea0a16c91d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.356674 5329 tablet_bootstrap.cc:654] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.356782 5288 tablet_bootstrap.cc:654] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.358660 5332 tablet_bootstrap.cc:492] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:51.359123 5288 tablet_bootstrap.cc:492] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:51.359215 5288 ts_tablet_manager.cc:1367] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:51.359712 5288 raft_consensus.cc:348] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.359849 5288 raft_consensus.cc:374] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.359890 5288 raft_consensus.cc:729] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.360034 5288 consensus_queue.cc:260] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.360330 5288 ts_tablet_manager.cc:1398] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:51.361169 5329 tablet_bootstrap.cc:492] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01: No bootstrap required, opened a new log
I20240502 14:00:51.361291 5329 ts_tablet_manager.cc:1367] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:00:51.361486 5332 tablet_bootstrap.cc:654] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.361884 5329 raft_consensus.cc:348] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.362023 5329 raft_consensus.cc:374] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.362058 5329 raft_consensus.cc:729] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.362237 5329 consensus_queue.cc:260] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.362816 5332 tablet_bootstrap.cc:492] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:51.362957 5332 ts_tablet_manager.cc:1367] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:51.363490 5332 raft_consensus.cc:348] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.363669 5332 raft_consensus.cc:374] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.363761 5332 raft_consensus.cc:729] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.362579 5329 ts_tablet_manager.cc:1398] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:51.364022 5332 consensus_queue.cc:260] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.368430 5332 ts_tablet_manager.cc:1398] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20240502 14:00:51.406607 5104 raft_consensus.cc:482] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:51.406737 5104 raft_consensus.cc:504] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:51.407133 5104 leader_election.cc:288] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:51.407575 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ad9695e69d49b68898b1b1b30e969b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:51.407560 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ad9695e69d49b68898b1b1b30e969b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:51.407732 2884 raft_consensus.cc:2454] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:51.407737 409 raft_consensus.cc:2454] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 0.
I20240502 14:00:51.408100 3219 leader_election.cc:302] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:51.408258 5104 raft_consensus.cc:2781] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:51.408305 5104 raft_consensus.cc:482] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:51.408365 5104 raft_consensus.cc:3037] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.409379 5104 raft_consensus.cc:504] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:51.409701 5104 leader_election.cc:288] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:51.410002 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ad9695e69d49b68898b1b1b30e969b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:51.410120 2884 raft_consensus.cc:3037] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.410147 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ad9695e69d49b68898b1b1b30e969b" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:51.410260 409 raft_consensus.cc:3037] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.411087 2884 raft_consensus.cc:2454] T e5ad9695e69d49b68898b1b1b30e969b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:51.411257 409 raft_consensus.cc:2454] T e5ad9695e69d49b68898b1b1b30e969b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
I20240502 14:00:51.411514 3218 leader_election.cc:302] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:51.411716 5104 raft_consensus.cc:2781] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:51.411898 5104 raft_consensus.cc:686] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:51.412050 5104 consensus_queue.cc:237] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:51.415167 32733 catalog_manager.cc:5484] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:51.643270 5104 consensus_queue.cc:1035] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:51.646453 5202 consensus_queue.cc:1035] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:51.692124 5199 raft_consensus.cc:482] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:51.692253 5199 raft_consensus.cc:504] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.692684 5199 leader_election.cc:288] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:51.693045 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be73835d3d04112a4b34ea0a16c91d2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485" is_pre_election: true
I20240502 14:00:51.692977 4972 consensus_queue.cc:1035] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:51.693078 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be73835d3d04112a4b34ea0a16c91d2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:51.693158 3150 raft_consensus.cc:2454] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:51.693230 3018 raft_consensus.cc:2454] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 0.
I20240502 14:00:51.693552 738 leader_election.cc:302] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:51.693723 5199 raft_consensus.cc:2781] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:51.693779 5199 raft_consensus.cc:482] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:51.693827 5199 raft_consensus.cc:3037] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.694903 5199 raft_consensus.cc:504] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.695282 5199 leader_election.cc:288] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Requested vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:51.696417 5342 consensus_queue.cc:1035] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:51.696671 3150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be73835d3d04112a4b34ea0a16c91d2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:51.696779 3150 raft_consensus.cc:3037] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.696969 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1be73835d3d04112a4b34ea0a16c91d2" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:51.697101 3018 raft_consensus.cc:3037] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:51.697871 3150 raft_consensus.cc:2454] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:51.698122 3018 raft_consensus.cc:2454] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 1.
I20240502 14:00:51.698462 738 leader_election.cc:302] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01; no voters:
I20240502 14:00:51.698771 5199 raft_consensus.cc:2781] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:51.698926 5199 raft_consensus.cc:686] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:51.699061 5199 consensus_queue.cc:237] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.701961 32733 catalog_manager.cc:5484] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:51.832342 32733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 14:00:51.836795 3018 raft_consensus.cc:1261] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:51.836920 3150 raft_consensus.cc:1261] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:51.837316 5262 consensus_queue.cc:1035] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:51.837504 5199 consensus_queue.cc:1035] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:51.842430 32730 catalog_manager.cc:5576] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509) does not have the latest schema for tablet 1be73835d3d04112a4b34ea0a16c91d2 (table rebalancer_dynamic_table [id=d1f75889e95049d3aaf74b5f91a96411]). Expected version 1 got 0
I20240502 14:00:51.842669 5351 tablet.cc:1722] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:51.843144 5350 tablet.cc:1722] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:51.843397 5351 tablet.cc:1620] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:51.846581 5350 tablet.cc:1620] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:51.848392 32730 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=d1f75889e95049d3aaf74b5f91a96411] alter complete (version 1)
I20240502 14:00:51.849172 32732 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet 1be73835d3d04112a4b34ea0a16c91d2 (table rebalancer_dynamic_table [id=d1f75889e95049d3aaf74b5f91a96411]). Expected version 1 got 0
I20240502 14:00:51.853169 32732 catalog_manager.cc:5576] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061) does not have the latest schema for tablet 1be73835d3d04112a4b34ea0a16c91d2 (table rebalancer_dynamic_table [id=d1f75889e95049d3aaf74b5f91a96411]). Expected version 1 got 0
I20240502 14:00:51.868026 5352 tablet.cc:1722] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 14:00:51.868382 5352 tablet.cc:1620] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:51.907683 5202 consensus_queue.cc:1035] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:51.911000 5104 consensus_queue.cc:1035] T e5ad9695e69d49b68898b1b1b30e969b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:51.953649 32732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38284:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 14:00:51.960284 5350 tablet.cc:1722] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:51.960364 5352 tablet.cc:1722] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:51.960573 5351 tablet.cc:1722] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 14:00:51.960667 5350 tablet.cc:1620] T 1be73835d3d04112a4b34ea0a16c91d2 P 478dfa53d7944f38b4e6cabee21b2d01: MemRowSet was empty: no flush needed.
I20240502 14:00:51.960758 5352 tablet.cc:1620] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860: MemRowSet was empty: no flush needed.
I20240502 14:00:51.960911 5351 tablet.cc:1620] T 1be73835d3d04112a4b34ea0a16c91d2 P 2b6796cf0e2e496e9dd641beb432f485: MemRowSet was empty: no flush needed.
I20240502 14:00:51.966245 32730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38284:
name: "rebalancer_extra_table_76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:51.966480 32733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 1ab7f53ff5194e848de277ee27020cc8: rebalancer_dynamic_table [id=d1f75889e95049d3aaf74b5f91a96411] alter complete (version 2)
I20240502 14:00:51.972604 3132 tablet_service.cc:1467] Processing CreateTablet for tablet adaf3fcf84de4223876c7601bdfd45ea (DEFAULT_TABLE table=rebalancer_extra_table_76 [id=4041aa4f8ea643298c2913693875a3a8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.972970 3132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adaf3fcf84de4223876c7601bdfd45ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.973479 3263 tablet_service.cc:1467] Processing CreateTablet for tablet 1eab750f98a6432494fa5351269e3052 (DEFAULT_TABLE table=rebalancer_extra_table_76 [id=4041aa4f8ea643298c2913693875a3a8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.973773 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eab750f98a6432494fa5351269e3052. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.973744 388 tablet_service.cc:1467] Processing CreateTablet for tablet f15dcc8632ab4a0096c2013e4fa74673 (DEFAULT_TABLE table=rebalancer_extra_table_76 [id=4041aa4f8ea643298c2913693875a3a8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.974002 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15dcc8632ab4a0096c2013e4fa74673. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.974085 520 tablet_service.cc:1467] Processing CreateTablet for tablet 1eab750f98a6432494fa5351269e3052 (DEFAULT_TABLE table=rebalancer_extra_table_76 [id=4041aa4f8ea643298c2913693875a3a8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.974359 520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eab750f98a6432494fa5351269e3052. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.975706 387 tablet_service.cc:1467] Processing CreateTablet for tablet adaf3fcf84de4223876c7601bdfd45ea (DEFAULT_TABLE table=rebalancer_extra_table_76 [id=4041aa4f8ea643298c2913693875a3a8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.975960 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adaf3fcf84de4223876c7601bdfd45ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.976457 5363 tablet_bootstrap.cc:492] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:51.977509 5362 tablet_bootstrap.cc:492] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:51.977614 2999 tablet_service.cc:1467] Processing CreateTablet for tablet adaf3fcf84de4223876c7601bdfd45ea (DEFAULT_TABLE table=rebalancer_extra_table_76 [id=4041aa4f8ea643298c2913693875a3a8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.977895 2997 tablet_service.cc:1467] Processing CreateTablet for tablet 1eab750f98a6432494fa5351269e3052 (DEFAULT_TABLE table=rebalancer_extra_table_76 [id=4041aa4f8ea643298c2913693875a3a8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.977914 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adaf3fcf84de4223876c7601bdfd45ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.979012 2866 tablet_service.cc:1467] Processing CreateTablet for tablet f15dcc8632ab4a0096c2013e4fa74673 (DEFAULT_TABLE table=rebalancer_extra_table_76 [id=4041aa4f8ea643298c2913693875a3a8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.979310 2866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15dcc8632ab4a0096c2013e4fa74673. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.979626 5361 tablet_bootstrap.cc:492] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:51.979779 653 tablet_service.cc:1467] Processing CreateTablet for tablet f15dcc8632ab4a0096c2013e4fa74673 (DEFAULT_TABLE table=rebalancer_extra_table_76 [id=4041aa4f8ea643298c2913693875a3a8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:51.980069 653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15dcc8632ab4a0096c2013e4fa74673. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.980662 5361 tablet_bootstrap.cc:654] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.981506 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eab750f98a6432494fa5351269e3052. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:51.984460 5364 tablet_bootstrap.cc:492] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:51.985118 5362 tablet_bootstrap.cc:654] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.985590 5364 tablet_bootstrap.cc:654] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.985869 5368 tablet_bootstrap.cc:492] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:51.986363 5362 tablet_bootstrap.cc:492] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013: No bootstrap required, opened a new log
I20240502 14:00:51.986470 5362 ts_tablet_manager.cc:1367] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:51.986758 5364 tablet_bootstrap.cc:492] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751: No bootstrap required, opened a new log
I20240502 14:00:51.986850 5364 ts_tablet_manager.cc:1367] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:51.986991 5362 raft_consensus.cc:348] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.987345 5364 raft_consensus.cc:348] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.987789 5364 raft_consensus.cc:374] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.987916 5364 raft_consensus.cc:729] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.988148 5364 consensus_queue.cc:260] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.988224 5368 tablet_bootstrap.cc:654] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.988574 5364 ts_tablet_manager.cc:1398] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:51.988979 5362 raft_consensus.cc:374] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.989099 5362 raft_consensus.cc:729] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.989306 5362 consensus_queue.cc:260] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.989710 5362 ts_tablet_manager.cc:1398] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:51.990269 5363 tablet_bootstrap.cc:654] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.990393 5368 tablet_bootstrap.cc:492] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:51.990515 5368 ts_tablet_manager.cc:1367] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:51.991011 5368 raft_consensus.cc:348] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.991153 5368 raft_consensus.cc:374] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.991205 5368 raft_consensus.cc:729] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.991287 5366 tablet_bootstrap.cc:492] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:51.991382 5368 consensus_queue.cc:260] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:51.992436 5366 tablet_bootstrap.cc:654] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.992581 5368 ts_tablet_manager.cc:1398] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:51.992748 5368 tablet_bootstrap.cc:492] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:51.993597 5367 tablet_bootstrap.cc:492] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:51.993839 5368 tablet_bootstrap.cc:654] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.994570 5363 tablet_bootstrap.cc:492] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:51.994877 5363 ts_tablet_manager.cc:1367] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 14:00:51.995430 5363 raft_consensus.cc:348] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:51.995575 5363 raft_consensus.cc:374] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:51.995628 5363 raft_consensus.cc:729] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:51.995787 5363 consensus_queue.cc:260] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:51.996173 5363 ts_tablet_manager.cc:1398] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:51.996230 5367 tablet_bootstrap.cc:654] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.997498 5363 tablet_bootstrap.cc:492] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:51.998394 5363 tablet_bootstrap.cc:654] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:51.998868 5368 tablet_bootstrap.cc:492] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860: No bootstrap required, opened a new log
I20240502 14:00:51.999670 5368 ts_tablet_manager.cc:1367] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:52.000133 5361 tablet_bootstrap.cc:492] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485: No bootstrap required, opened a new log
I20240502 14:00:52.000233 5361 ts_tablet_manager.cc:1367] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 14:00:52.000205 5368 raft_consensus.cc:348] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.000311 5368 raft_consensus.cc:374] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:52.000342 5368 raft_consensus.cc:729] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: FOLLOWER
I20240502 14:00:52.000491 5368 consensus_queue.cc:260] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.000687 5361 raft_consensus.cc:348] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.001310 5361 raft_consensus.cc:374] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:52.001367 5361 raft_consensus.cc:729] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: FOLLOWER
I20240502 14:00:52.001510 5361 consensus_queue.cc:260] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.001812 5361 ts_tablet_manager.cc:1398] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:52.002162 5368 ts_tablet_manager.cc:1398] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:52.003949 5366 tablet_bootstrap.cc:492] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5: No bootstrap required, opened a new log
I20240502 14:00:52.004994 5363 tablet_bootstrap.cc:492] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e: No bootstrap required, opened a new log
I20240502 14:00:52.005097 5363 ts_tablet_manager.cc:1367] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:52.005754 5363 raft_consensus.cc:348] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.005894 5363 raft_consensus.cc:374] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:52.005947 5363 raft_consensus.cc:729] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: FOLLOWER
I20240502 14:00:52.006098 5363 consensus_queue.cc:260] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.006489 5366 ts_tablet_manager.cc:1367] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:00:52.006978 5366 raft_consensus.cc:348] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.007098 5366 raft_consensus.cc:374] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:52.007318 5366 raft_consensus.cc:729] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: FOLLOWER
I20240502 14:00:52.007498 5366 consensus_queue.cc:260] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.008240 5363 ts_tablet_manager.cc:1398] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:52.010591 5366 ts_tablet_manager.cc:1398] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:52.012691 5283 raft_consensus.cc:482] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:52.012789 5283 raft_consensus.cc:504] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.013283 5283 leader_election.cc:288] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:52.013741 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15dcc8632ab4a0096c2013e4fa74673" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:52.013861 2884 raft_consensus.cc:2454] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 0.
I20240502 14:00:52.014171 344 leader_election.cc:302] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:52.014438 5283 raft_consensus.cc:2781] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:52.014496 5283 raft_consensus.cc:482] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:52.014531 5283 raft_consensus.cc:3037] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:52.014515 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15dcc8632ab4a0096c2013e4fa74673" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:00:52.015712 5283 raft_consensus.cc:504] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.016064 5283 leader_election.cc:288] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:52.016610 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15dcc8632ab4a0096c2013e4fa74673" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:52.016711 2884 raft_consensus.cc:3037] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:52.017688 2884 raft_consensus.cc:2454] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:00:52.017993 344 leader_election.cc:302] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, f003b732dd0f46958262b42f8d20305e; no voters:
W20240502 14:00:52.018271 341 leader_election.cc:341] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:00:52.018355 5096 raft_consensus.cc:2781] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:52.018417 5096 raft_consensus.cc:686] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:00:52.018448 674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15dcc8632ab4a0096c2013e4fa74673" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:52.018551 5096 consensus_queue.cc:237] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
W20240502 14:00:52.019996 341 leader_election.cc:341] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:00:52.020819 5367 tablet_bootstrap.cc:492] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9: No bootstrap required, opened a new log
I20240502 14:00:52.021371 32731 catalog_manager.cc:5484] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 0 to 1, leader changed from <none> to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:52.021975 5367 ts_tablet_manager.cc:1367] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20240502 14:00:52.022496 5367 raft_consensus.cc:348] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.022686 5367 raft_consensus.cc:374] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:52.024806 5367 raft_consensus.cc:729] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: FOLLOWER
I20240502 14:00:52.025038 5367 consensus_queue.cc:260] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.026127 5367 ts_tablet_manager.cc:1398] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:52.116699 5342 raft_consensus.cc:482] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:52.116849 5342 raft_consensus.cc:504] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.117273 5342 leader_election.cc:288] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:52.117619 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adaf3fcf84de4223876c7601bdfd45ea" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:00:52.117769 3018 raft_consensus.cc:2454] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:52.117784 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adaf3fcf84de4223876c7601bdfd45ea" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:52.117949 409 raft_consensus.cc:2454] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 0.
I20240502 14:00:52.118110 3084 leader_election.cc:302] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 452ecbc085e54db49c3b01c20a96f860; no voters:
I20240502 14:00:52.118319 5342 raft_consensus.cc:2781] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:52.118369 5342 raft_consensus.cc:482] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:52.118415 5342 raft_consensus.cc:3037] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:52.119451 5342 raft_consensus.cc:504] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.119764 5342 leader_election.cc:288] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:52.120146 3018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adaf3fcf84de4223876c7601bdfd45ea" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:52.120163 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adaf3fcf84de4223876c7601bdfd45ea" candidate_uuid: "2b6796cf0e2e496e9dd641beb432f485" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:52.120285 3018 raft_consensus.cc:3037] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:52.120283 409 raft_consensus.cc:3037] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:52.121451 409 raft_consensus.cc:2454] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:52.121464 3018 raft_consensus.cc:2454] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6796cf0e2e496e9dd641beb432f485 in term 1.
I20240502 14:00:52.121766 3086 leader_election.cc:302] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:52.121925 5342 raft_consensus.cc:2781] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:52.122022 5342 raft_consensus.cc:686] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Becoming Leader. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Running, Role: LEADER
I20240502 14:00:52.122169 5342 consensus_queue.cc:237] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.124862 32731 catalog_manager.cc:5484] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135). New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:52.235311 5095 raft_consensus.cc:482] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:52.235453 5095 raft_consensus.cc:504] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:52.235894 5095 leader_election.cc:288] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:52.236225 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eab750f98a6432494fa5351269e3052" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:00:52.236366 3283 raft_consensus.cc:2454] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:52.236639 2952 leader_election.cc:302] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:52.236642 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eab750f98a6432494fa5351269e3052" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:00:52.236766 542 raft_consensus.cc:2454] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 0.
I20240502 14:00:52.236951 5095 raft_consensus.cc:2781] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:52.237018 5095 raft_consensus.cc:482] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:52.237058 5095 raft_consensus.cc:3037] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:52.238185 5095 raft_consensus.cc:504] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:52.238584 5095 leader_election.cc:288] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Requested vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:52.238905 542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eab750f98a6432494fa5351269e3052" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:52.238924 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eab750f98a6432494fa5351269e3052" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:52.239007 542 raft_consensus.cc:3037] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:52.239025 3283 raft_consensus.cc:3037] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:52.239930 542 raft_consensus.cc:2454] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:52.239996 3283 raft_consensus.cc:2454] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:00:52.240227 2951 leader_election.cc:302] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:00:52.240446 5095 raft_consensus.cc:2781] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:52.240522 5095 raft_consensus.cc:686] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:52.240651 5095 consensus_queue.cc:237] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:00:52.243184 32731 catalog_manager.cc:5484] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:52.425748 5098 raft_consensus.cc:482] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:52.425870 5098 raft_consensus.cc:504] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:00:52.426415 5098 leader_election.cc:288] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:52.426785 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15dcc8632ab4a0096c2013e4fa74673" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:00:52.426797 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15dcc8632ab4a0096c2013e4fa74673" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:00:52.426988 2886 raft_consensus.cc:2379] T f15dcc8632ab4a0096c2013e4fa74673 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58107b611f8f475a910a6fd54dfd1fb9 in current term 1: Already voted for candidate f003b732dd0f46958262b42f8d20305e in this term.
I20240502 14:00:52.427425 609 leader_election.cc:302] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9; no voters: 4bcd7712aa844d98977a7d7e605c31f5, f003b732dd0f46958262b42f8d20305e
I20240502 14:00:52.427656 5098 raft_consensus.cc:3037] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:52.428591 5098 raft_consensus.cc:2726] T f15dcc8632ab4a0096c2013e4fa74673 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 14:00:52.506980 5096 consensus_queue.cc:1035] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:52.526100 5283 consensus_queue.cc:1035] T f15dcc8632ab4a0096c2013e4fa74673 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:52.530373 5299 consensus_queue.cc:1035] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:52.538909 5299 consensus_queue.cc:1035] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:52.626405 5095 consensus_queue.cc:1035] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:52.638864 3132 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 14:00:52.651329 5282 consensus_queue.cc:1035] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:52.653334 653 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
I20240502 14:00:52.653550 2999 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 14:00:52.657089 785 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 14:00:52.675567 520 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 14:00:52.697162 2866 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 14:00:52.712286 3263 tablet_service.cc:1429] Tablet server has 26 leaders and 0 scanners
I20240502 14:00:52.718194 387 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 14:00:52.968379 3132 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 14:00:52.975433 653 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
I20240502 14:00:52.976697 2866 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 14:00:52.985406 387 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 14:00:52.985431 520 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 14:00:52.988678 3263 tablet_service.cc:1429] Tablet server has 26 leaders and 0 scanners
I20240502 14:00:52.990784 2999 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 14:00:53.013494 785 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 14:00:53.312515 3263 tablet_service.cc:1429] Tablet server has 26 leaders and 0 scanners
I20240502 14:00:53.317709 387 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 14:00:53.318550 520 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 14:00:53.327744 2866 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 14:00:53.328305 2999 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 14:00:53.335366 3132 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 14:00:53.341996 653 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
I20240502 14:00:53.355530 785 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 14:00:53.471679 805 consensus_queue.cc:237] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:00:53.473276 541 raft_consensus.cc:1261] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.473678 3282 raft_consensus.cc:1261] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.474154 5430 consensus_queue.cc:1035] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.474350 5582 consensus_queue.cc:1035] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.476321 5430 raft_consensus.cc:2932] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:53.476760 3282 raft_consensus.cc:2932] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
W20240502 14:00:53.478355 741 consensus_peers.cc:485] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 -> Peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): Couldn't send request to peer 4bcd7712aa844d98977a7d7e605c31f5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 72395d91ff48409693431748b3efeb6d. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.480587 32728 catalog_manager.cc:5484] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index -1 to 2, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.483057 541 raft_consensus.cc:2932] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:53.500954 3150 consensus_queue.cc:237] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:53.502663 2886 raft_consensus.cc:1261] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.503098 805 raft_consensus.cc:1261] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.503064 5342 consensus_queue.cc:1035] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:53.503597 3084 consensus_peers.cc:485] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbae9f2a88db4a92b89cc12dfb837ad2. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.503567 5342 consensus_queue.cc:1035] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.506199 5299 raft_consensus.cc:2932] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:53.506562 805 raft_consensus.cc:2932] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:53.508944 2886 raft_consensus.cc:2932] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:53.510083 32729 catalog_manager.cc:5484] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.521874 3150 consensus_queue.cc:237] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:53.523330 3019 raft_consensus.cc:1261] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.523350 2886 raft_consensus.cc:1261] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.523731 5380 consensus_queue.cc:1035] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:53.524034 3086 consensus_peers.cc:485] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4fc8d5ad25334f0a8c2efd17b5779330. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.524137 5342 consensus_queue.cc:1035] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.525918 5597 raft_consensus.cc:2932] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.526319 2886 raft_consensus.cc:2932] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.527221 3019 raft_consensus.cc:2932] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.530628 32728 catalog_manager.cc:5484] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.541416 672 consensus_queue.cc:237] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:53.542851 409 raft_consensus.cc:1261] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:53.542928 541 raft_consensus.cc:1261] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:53.543329 5563 consensus_queue.cc:1035] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:00:53.543550 609 consensus_peers.cc:485] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f9b1351891d4690b67f8eef000a902b. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.543556 5098 consensus_queue.cc:1035] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:53.545316 5293 raft_consensus.cc:2932] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.545737 409 raft_consensus.cc:2932] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.547430 541 raft_consensus.cc:2932] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.550664 32729 catalog_manager.cc:5484] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.560945 2886 consensus_queue.cc:237] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:53.562371 805 raft_consensus.cc:1261] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.563000 3019 raft_consensus.cc:1261] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:53.563154 2820 consensus_peers.cc:485] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 909bc2eb63544db0b4b3a0764be7d4a1. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.563438 5620 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.563684 5358 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.565510 5620 raft_consensus.cc:2932] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:53.566357 3019 raft_consensus.cc:2932] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:53.566958 805 raft_consensus.cc:2932] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:53.569397 32726 catalog_manager.cc:5484] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.579258 672 consensus_queue.cc:237] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:53.580686 409 raft_consensus.cc:1261] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:53.580785 805 raft_consensus.cc:1261] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:53.580940 606 consensus_peers.cc:485] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): Couldn't send request to peer 452ecbc085e54db49c3b01c20a96f860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 95101f5c218746eab5095b984181a5fc. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.581248 5098 consensus_queue.cc:1035] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:53.581429 5563 consensus_queue.cc:1035] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:53.583186 5098 raft_consensus.cc:2932] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:53.583665 409 raft_consensus.cc:2932] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:53.587090 32729 catalog_manager.cc:5484] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.593389 805 raft_consensus.cc:2932] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:53.601341 3019 consensus_queue.cc:237] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:53.602977 2886 raft_consensus.cc:1261] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.603209 672 raft_consensus.cc:1261] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.603368 5095 consensus_queue.cc:1035] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.603698 5095 consensus_queue.cc:1035] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:53.603973 2953 consensus_peers.cc:485] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6423e05535aa41d5923f570c7b73f61c. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.606691 5282 raft_consensus.cc:2932] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.607035 2886 raft_consensus.cc:2932] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.610560 32728 catalog_manager.cc:5484] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.625283 672 consensus_queue.cc:237] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:53.627027 541 raft_consensus.cc:1261] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:53.627094 609 consensus_peers.cc:485] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5f321621da5d481b8d1d2b25ddf4b395. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.627745 5293 consensus_queue.cc:1035] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:53.627915 805 raft_consensus.cc:1261] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:53.628453 5293 consensus_queue.cc:1035] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:53.630247 5098 raft_consensus.cc:2932] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.631314 805 raft_consensus.cc:2932] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.631326 541 raft_consensus.cc:2932] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.634117 32726 catalog_manager.cc:5484] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.645112 541 consensus_queue.cc:237] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:53.646812 3019 raft_consensus.cc:1261] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.646826 2886 raft_consensus.cc:1261] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:53.646895 475 consensus_peers.cc:485] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 093babce464c4152ab6ade948dc7a616. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.647257 5655 consensus_queue.cc:1035] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.647485 5295 consensus_queue.cc:1035] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.649283 5575 raft_consensus.cc:2932] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:53.649713 2886 raft_consensus.cc:2932] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:53.649806 3019 raft_consensus.cc:2932] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:53.653846 32728 catalog_manager.cc:5484] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.664994 3282 consensus_queue.cc:237] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:53.666558 409 raft_consensus.cc:1261] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.666633 3019 raft_consensus.cc:1261] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:53.666986 3219 consensus_peers.cc:485] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b985ba4b9f534827919cbb24daa4de0e. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.667093 5104 consensus_queue.cc:1035] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.667274 5354 consensus_queue.cc:1035] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.669128 5104 raft_consensus.cc:2932] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:53.669682 409 raft_consensus.cc:2932] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:53.671957 3019 raft_consensus.cc:2932] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:53.672811 32732 catalog_manager.cc:5484] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.686559 2886 consensus_queue.cc:237] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:53.688196 541 raft_consensus.cc:1261] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:53.688357 2819 consensus_peers.cc:485] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4aa562f43a154941a6e645e04757870d. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.688361 409 raft_consensus.cc:1261] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.688647 5358 consensus_queue.cc:1035] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.688822 5620 consensus_queue.cc:1035] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.690773 5620 raft_consensus.cc:2932] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:53.692169 541 raft_consensus.cc:2932] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:53.693734 409 raft_consensus.cc:2932] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:53.694559 32732 catalog_manager.cc:5484] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.705045 3019 consensus_queue.cc:237] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:53.706535 409 raft_consensus.cc:1261] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.706712 2886 raft_consensus.cc:1261] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.707001 5640 consensus_queue.cc:1035] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:53.707212 2954 consensus_peers.cc:485] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c47fb9f1470349b3be140e7b70811e7b. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.707221 5637 consensus_queue.cc:1035] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.709362 5637 raft_consensus.cc:2932] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.710456 2886 raft_consensus.cc:2932] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.710848 409 raft_consensus.cc:2932] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.714433 32729 catalog_manager.cc:5484] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.724563 3282 consensus_queue.cc:237] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:53.725915 2886 raft_consensus.cc:1261] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.725915 3019 raft_consensus.cc:1261] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.726370 5354 consensus_queue.cc:1035] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:53.726610 3219 consensus_peers.cc:485] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6346d5ea09834bddb483da26f9bce936. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.726722 5202 consensus_queue.cc:1035] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.728551 5202 raft_consensus.cc:2932] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:53.729344 3019 raft_consensus.cc:2932] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:53.729471 2886 raft_consensus.cc:2932] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:53.732829 32729 catalog_manager.cc:5484] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:53.744921 3019 consensus_queue.cc:237] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:53.746368 3150 raft_consensus.cc:1261] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:53.747042 2952 consensus_peers.cc:485] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 923a5d2ccd0448b381d88ced69ae8fe2. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.748039 2886 raft_consensus.cc:1261] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.748127 5640 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.748512 5095 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.750349 5095 raft_consensus.cc:2932] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:53.750685 3150 raft_consensus.cc:2932] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:53.750747 2886 raft_consensus.cc:2932] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:53.753767 32728 catalog_manager.cc:5484] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.763289 672 consensus_queue.cc:237] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:53.764976 541 raft_consensus.cc:1261] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.765051 3019 raft_consensus.cc:1261] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:53.765283 608 consensus_peers.cc:485] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a658bde2687344beabf4d8a0ff8988e2. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.765430 5098 consensus_queue.cc:1035] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.765689 5563 consensus_queue.cc:1035] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.767376 5293 raft_consensus.cc:2932] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.767871 541 raft_consensus.cc:2932] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.767901 3019 raft_consensus.cc:2932] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.771855 32729 catalog_manager.cc:5484] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.783385 409 consensus_queue.cc:237] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:53.785060 672 raft_consensus.cc:1261] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:53.785099 541 raft_consensus.cc:1261] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:53.785189 344 consensus_peers.cc:485] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ce81659e36874bb89da4d2268eb40aea. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.785482 5715 consensus_queue.cc:1035] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:53.785848 5714 consensus_queue.cc:1035] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:53.787454 5096 raft_consensus.cc:2932] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.787886 672 raft_consensus.cc:2932] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.791266 32728 catalog_manager.cc:5484] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.795570 542 raft_consensus.cc:2932] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.799050 3150 consensus_queue.cc:237] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
W20240502 14:00:53.800779 3087 consensus_peers.cc:485] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3bcaf05a01194e919efca9cc08d61c8f. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.801082 3018 raft_consensus.cc:1261] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.801452 5597 consensus_queue.cc:1035] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.801856 2886 raft_consensus.cc:1261] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.802300 5597 consensus_queue.cc:1035] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.803501 5299 raft_consensus.cc:2932] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:53.804054 3018 raft_consensus.cc:2932] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:53.804716 2886 raft_consensus.cc:2932] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:53.807502 32729 catalog_manager.cc:5484] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:53.817590 805 consensus_queue.cc:237] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:53.819239 672 raft_consensus.cc:1261] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.819239 3018 raft_consensus.cc:1261] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:53.819437 738 consensus_peers.cc:485] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56a192eba0af4486a212972ffdfacf9d. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.819623 5582 consensus_queue.cc:1035] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.819792 5429 consensus_queue.cc:1035] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.821687 5582 raft_consensus.cc:2932] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:53.822116 3018 raft_consensus.cc:2932] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:53.822540 672 raft_consensus.cc:2932] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:53.825402 32732 catalog_manager.cc:5484] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:53.835538 3282 consensus_queue.cc:237] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:53.837062 3018 raft_consensus.cc:1261] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.837091 805 raft_consensus.cc:1261] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.837495 5104 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.837661 5202 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:53.837878 3218 consensus_peers.cc:485] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 094aa25b08b8498f9ae4493cafd289e3. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.839504 5354 raft_consensus.cc:2932] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.843698 805 raft_consensus.cc:2932] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.844517 3018 raft_consensus.cc:2932] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:53.844802 32728 catalog_manager.cc:5484] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.856789 542 consensus_queue.cc:237] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:53.858620 672 raft_consensus.cc:1261] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.859205 5575 consensus_queue.cc:1035] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:53.860617 477 consensus_peers.cc:485] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fee3c05d88514648b8291e01eda2db2b. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:53.860966 2886 raft_consensus.cc:1261] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:53.861495 5655 consensus_queue.cc:1035] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:53.861871 5655 raft_consensus.cc:2932] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.862246 672 raft_consensus.cc:2932] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.864521 2886 raft_consensus.cc:2932] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:53.866578 32732 catalog_manager.cc:5484] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:53.921018 5790 ts_tablet_manager.cc:902] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5: Initiating tablet copy from peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:53.921859 5790 tablet_copy_client.cc:308] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.132:33443
I20240502 14:00:53.922669 826 tablet_copy_service.cc:140] P 478dfa53d7944f38b4e6cabee21b2d01: Received BeginTabletCopySession request for tablet 72395d91ff48409693431748b3efeb6d from peer 4bcd7712aa844d98977a7d7e605c31f5 ({username='slave'} at 127.26.25.133:51615)
I20240502 14:00:53.922827 826 tablet_copy_service.cc:161] P 478dfa53d7944f38b4e6cabee21b2d01: Beginning new tablet copy session on tablet 72395d91ff48409693431748b3efeb6d from peer 4bcd7712aa844d98977a7d7e605c31f5 at {username='slave'} at 127.26.25.133:51615: session id = 4bcd7712aa844d98977a7d7e605c31f5-72395d91ff48409693431748b3efeb6d
I20240502 14:00:53.924461 826 tablet_copy_source_session.cc:205] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:53.927491 5790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72395d91ff48409693431748b3efeb6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:53.930199 5790 tablet_copy_client.cc:783] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:53.930415 5790 tablet_copy_client.cc:647] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:53.932583 5790 tablet_copy_client.cc:523] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:53.933987 5790 tablet_bootstrap.cc:492] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:53.939049 5790 tablet_bootstrap.cc:492] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:53.939366 5790 tablet_bootstrap.cc:492] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap complete.
I20240502 14:00:53.939564 5790 ts_tablet_manager.cc:1367] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20240502 14:00:53.940052 5790 raft_consensus.cc:348] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:00:53.940214 5790 raft_consensus.cc:729] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: LEARNER
I20240502 14:00:53.940330 5790 consensus_queue.cc:260] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:00:53.940768 5790 ts_tablet_manager.cc:1398] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:00:53.942047 826 tablet_copy_service.cc:342] P 478dfa53d7944f38b4e6cabee21b2d01: Request end of tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-72395d91ff48409693431748b3efeb6d received from {username='slave'} at 127.26.25.133:51615
I20240502 14:00:53.942158 826 tablet_copy_service.cc:433] P 478dfa53d7944f38b4e6cabee21b2d01: ending tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-72395d91ff48409693431748b3efeb6d on tablet 72395d91ff48409693431748b3efeb6d with peer 4bcd7712aa844d98977a7d7e605c31f5
I20240502 14:00:53.983929 5836 ts_tablet_manager.cc:902] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:53.984817 5836 tablet_copy_client.cc:308] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:00:53.985877 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet bbae9f2a88db4a92b89cc12dfb837ad2 from peer 58107b611f8f475a910a6fd54dfd1fb9 ({username='slave'} at 127.26.25.131:33211)
I20240502 14:00:53.986033 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet bbae9f2a88db4a92b89cc12dfb837ad2 from peer 58107b611f8f475a910a6fd54dfd1fb9 at {username='slave'} at 127.26.25.131:33211: session id = 58107b611f8f475a910a6fd54dfd1fb9-bbae9f2a88db4a92b89cc12dfb837ad2
I20240502 14:00:53.987630 3172 tablet_copy_source_session.cc:205] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:53.989251 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbae9f2a88db4a92b89cc12dfb837ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:53.993109 5836 tablet_copy_client.cc:783] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:53.993304 5836 tablet_copy_client.cc:647] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:53.995534 5836 tablet_copy_client.cc:523] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:53.997256 5836 tablet_bootstrap.cc:492] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:54.000128 5851 ts_tablet_manager.cc:902] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:54.001237 5851 tablet_copy_client.cc:308] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:54.002566 694 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 3f9b1351891d4690b67f8eef000a902b from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:37651)
I20240502 14:00:54.002732 694 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 3f9b1351891d4690b67f8eef000a902b from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:37651: session id = 2b6796cf0e2e496e9dd641beb432f485-3f9b1351891d4690b67f8eef000a902b
I20240502 14:00:54.003871 694 tablet_copy_source_session.cc:205] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.005177 5851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f9b1351891d4690b67f8eef000a902b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.007450 5836 tablet_bootstrap.cc:492] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.007737 5836 tablet_bootstrap.cc:492] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap complete.
I20240502 14:00:54.007947 5836 ts_tablet_manager.cc:1367] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20240502 14:00:54.008442 5836 raft_consensus.cc:348] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:54.008569 5836 raft_consensus.cc:729] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: LEARNER
I20240502 14:00:54.008687 5836 consensus_queue.cc:260] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:54.008817 5851 tablet_copy_client.cc:783] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.008996 5851 tablet_copy_client.cc:647] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.009029 5836 ts_tablet_manager.cc:1398] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:54.009593 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-bbae9f2a88db4a92b89cc12dfb837ad2 received from {username='slave'} at 127.26.25.131:33211
I20240502 14:00:54.009666 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-bbae9f2a88db4a92b89cc12dfb837ad2 on tablet bbae9f2a88db4a92b89cc12dfb837ad2 with peer 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:00:54.024237 5851 tablet_copy_client.cc:523] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.026419 5851 tablet_bootstrap.cc:492] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:54.040050 672 raft_consensus.cc:2932] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:54.057860 5868 ts_tablet_manager.cc:902] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:54.058697 5868 tablet_copy_client.cc:308] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35251
I20240502 14:00:54.059535 562 tablet_copy_service.cc:140] P c351a89a5fe34e788b9d8064fbffa751: Received BeginTabletCopySession request for tablet 093babce464c4152ab6ade948dc7a616 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:53551)
I20240502 14:00:54.059692 562 tablet_copy_service.cc:161] P c351a89a5fe34e788b9d8064fbffa751: Beginning new tablet copy session on tablet 093babce464c4152ab6ade948dc7a616 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:53551: session id = da3f8160557e454f8547f45e80c2f013-093babce464c4152ab6ade948dc7a616
I20240502 14:00:54.061915 562 tablet_copy_source_session.cc:205] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.063807 5868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 093babce464c4152ab6ade948dc7a616. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.066802 5868 tablet_copy_client.cc:783] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.066998 5868 tablet_copy_client.cc:647] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.070111 5868 tablet_copy_client.cc:523] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.070442 5851 tablet_bootstrap.cc:492] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.070822 5851 tablet_bootstrap.cc:492] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:54.071028 5851 ts_tablet_manager.cc:1367] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.045s user 0.031s sys 0.000s
I20240502 14:00:54.071532 5851 raft_consensus.cc:348] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:54.071660 5851 raft_consensus.cc:729] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:54.071790 5851 consensus_queue.cc:260] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:54.072376 5868 tablet_bootstrap.cc:492] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:54.072712 5873 ts_tablet_manager.cc:902] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:54.073078 5851 ts_tablet_manager.cc:1398] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:54.073678 694 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-3f9b1351891d4690b67f8eef000a902b received from {username='slave'} at 127.26.25.135:37651
I20240502 14:00:54.073786 694 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-3f9b1351891d4690b67f8eef000a902b on tablet 3f9b1351891d4690b67f8eef000a902b with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:54.075039 5873 tablet_copy_client.cc:308] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:00:54.075727 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet 4fc8d5ad25334f0a8c2efd17b5779330 from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:55289)
I20240502 14:00:54.075939 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet 4fc8d5ad25334f0a8c2efd17b5779330 from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:55289: session id = f003b732dd0f46958262b42f8d20305e-4fc8d5ad25334f0a8c2efd17b5779330
I20240502 14:00:54.076895 3172 tablet_copy_source_session.cc:205] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.077956 5868 tablet_bootstrap.cc:492] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.078226 5868 tablet_bootstrap.cc:492] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:00:54.078409 5873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc8d5ad25334f0a8c2efd17b5779330. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.078413 5868 ts_tablet_manager.cc:1367] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20240502 14:00:54.078871 5868 raft_consensus.cc:348] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:54.078996 5868 raft_consensus.cc:729] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:00:54.079095 5868 consensus_queue.cc:260] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:54.080296 5868 ts_tablet_manager.cc:1398] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20240502 14:00:54.080879 5873 tablet_copy_client.cc:783] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.081076 5873 tablet_copy_client.cc:647] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.081369 562 tablet_copy_service.cc:342] P c351a89a5fe34e788b9d8064fbffa751: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-093babce464c4152ab6ade948dc7a616 received from {username='slave'} at 127.26.25.136:53551
I20240502 14:00:54.081465 562 tablet_copy_service.cc:433] P c351a89a5fe34e788b9d8064fbffa751: ending tablet copy session da3f8160557e454f8547f45e80c2f013-093babce464c4152ab6ade948dc7a616 on tablet 093babce464c4152ab6ade948dc7a616 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:00:54.084165 5873 tablet_copy_client.cc:523] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.087378 5873 tablet_bootstrap.cc:492] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:54.090806 5880 ts_tablet_manager.cc:902] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:54.091408 5880 tablet_copy_client.cc:308] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.134:39061
I20240502 14:00:54.091420 5873 tablet_bootstrap.cc:492] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.091720 5873 tablet_bootstrap.cc:492] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:00:54.091926 5873 ts_tablet_manager.cc:1367] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.002s
I20240502 14:00:54.092331 3039 tablet_copy_service.cc:140] P 452ecbc085e54db49c3b01c20a96f860: Received BeginTabletCopySession request for tablet 6423e05535aa41d5923f570c7b73f61c from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:35347)
I20240502 14:00:54.092351 5873 raft_consensus.cc:348] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:54.092476 5873 raft_consensus.cc:729] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:00:54.092499 3039 tablet_copy_service.cc:161] P 452ecbc085e54db49c3b01c20a96f860: Beginning new tablet copy session on tablet 6423e05535aa41d5923f570c7b73f61c from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:35347: session id = f003b732dd0f46958262b42f8d20305e-6423e05535aa41d5923f570c7b73f61c
I20240502 14:00:54.092621 5873 consensus_queue.cc:260] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:54.092933 5873 ts_tablet_manager.cc:1398] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:54.093456 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-4fc8d5ad25334f0a8c2efd17b5779330 received from {username='slave'} at 127.26.25.129:55289
I20240502 14:00:54.093518 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session f003b732dd0f46958262b42f8d20305e-4fc8d5ad25334f0a8c2efd17b5779330 on tablet 4fc8d5ad25334f0a8c2efd17b5779330 with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:00:54.094106 3039 tablet_copy_source_session.cc:205] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.095607 5880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6423e05535aa41d5923f570c7b73f61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.097604 5880 tablet_copy_client.cc:783] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.097860 5880 tablet_copy_client.cc:647] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.100402 5880 tablet_copy_client.cc:523] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.101629 5880 tablet_bootstrap.cc:492] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:54.104516 5868 ts_tablet_manager.cc:902] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:54.105053 5868 tablet_copy_client.cc:308] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:00:54.105384 5880 tablet_bootstrap.cc:492] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.105670 5880 tablet_bootstrap.cc:492] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:00:54.105690 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 909bc2eb63544db0b4b3a0764be7d4a1 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:39679)
I20240502 14:00:54.105837 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 909bc2eb63544db0b4b3a0764be7d4a1 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:39679: session id = da3f8160557e454f8547f45e80c2f013-909bc2eb63544db0b4b3a0764be7d4a1
I20240502 14:00:54.105849 5880 ts_tablet_manager.cc:1367] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 14:00:54.106714 5880 raft_consensus.cc:348] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:54.106855 5880 raft_consensus.cc:729] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:00:54.106974 5880 consensus_queue.cc:260] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:54.107275 2906 tablet_copy_source_session.cc:205] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.107775 5880 ts_tablet_manager.cc:1398] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 14:00:54.108516 3039 tablet_copy_service.cc:342] P 452ecbc085e54db49c3b01c20a96f860: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-6423e05535aa41d5923f570c7b73f61c received from {username='slave'} at 127.26.25.129:35347
I20240502 14:00:54.108603 3039 tablet_copy_service.cc:433] P 452ecbc085e54db49c3b01c20a96f860: ending tablet copy session f003b732dd0f46958262b42f8d20305e-6423e05535aa41d5923f570c7b73f61c on tablet 6423e05535aa41d5923f570c7b73f61c with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:00:54.108870 5868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 909bc2eb63544db0b4b3a0764be7d4a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.109505 5890 ts_tablet_manager.cc:902] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:54.110399 5890 tablet_copy_client.cc:308] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:54.110983 694 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 95101f5c218746eab5095b984181a5fc from peer 452ecbc085e54db49c3b01c20a96f860 ({username='slave'} at 127.26.25.134:41957)
I20240502 14:00:54.111093 694 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 95101f5c218746eab5095b984181a5fc from peer 452ecbc085e54db49c3b01c20a96f860 at {username='slave'} at 127.26.25.134:41957: session id = 452ecbc085e54db49c3b01c20a96f860-95101f5c218746eab5095b984181a5fc
I20240502 14:00:54.111292 5868 tablet_copy_client.cc:783] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.111989 694 tablet_copy_source_session.cc:205] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.112639 5868 tablet_copy_client.cc:647] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.114032 5890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95101f5c218746eab5095b984181a5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.114720 5868 tablet_copy_client.cc:523] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.114920 5851 ts_tablet_manager.cc:902] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:54.115365 5851 tablet_copy_client.cc:308] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.134:39061
I20240502 14:00:54.115809 3039 tablet_copy_service.cc:140] P 452ecbc085e54db49c3b01c20a96f860: Received BeginTabletCopySession request for tablet c47fb9f1470349b3be140e7b70811e7b from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:48771)
I20240502 14:00:54.115938 3039 tablet_copy_service.cc:161] P 452ecbc085e54db49c3b01c20a96f860: Beginning new tablet copy session on tablet c47fb9f1470349b3be140e7b70811e7b from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:48771: session id = 2b6796cf0e2e496e9dd641beb432f485-c47fb9f1470349b3be140e7b70811e7b
I20240502 14:00:54.116379 5868 tablet_bootstrap.cc:492] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:54.117009 5890 tablet_copy_client.cc:783] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.117262 5890 tablet_copy_client.cc:647] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.117281 3039 tablet_copy_source_session.cc:205] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.118223 5851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c47fb9f1470349b3be140e7b70811e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.120539 5851 tablet_copy_client.cc:783] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.120698 5851 tablet_copy_client.cc:647] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.120909 5890 tablet_copy_client.cc:523] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.121032 5868 tablet_bootstrap.cc:492] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.121328 5868 tablet_bootstrap.cc:492] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:00:54.121516 5868 ts_tablet_manager.cc:1367] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20240502 14:00:54.121883 5851 tablet_copy_client.cc:523] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.122022 5868 raft_consensus.cc:348] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:54.122133 5868 raft_consensus.cc:729] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:00:54.122223 5868 consensus_queue.cc:260] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:54.122437 5890 tablet_bootstrap.cc:492] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:54.122519 5868 ts_tablet_manager.cc:1398] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:54.123093 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-909bc2eb63544db0b4b3a0764be7d4a1 received from {username='slave'} at 127.26.25.136:39679
I20240502 14:00:54.123194 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session da3f8160557e454f8547f45e80c2f013-909bc2eb63544db0b4b3a0764be7d4a1 on tablet 909bc2eb63544db0b4b3a0764be7d4a1 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:00:54.123313 5851 tablet_bootstrap.cc:492] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:54.126655 5851 tablet_bootstrap.cc:492] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.126863 5851 tablet_bootstrap.cc:492] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:54.126982 5851 ts_tablet_manager.cc:1367] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:54.127449 5851 raft_consensus.cc:348] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:54.127579 5851 raft_consensus.cc:729] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:54.127677 5851 consensus_queue.cc:260] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:54.129195 5851 ts_tablet_manager.cc:1398] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:54.129659 3039 tablet_copy_service.cc:342] P 452ecbc085e54db49c3b01c20a96f860: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-c47fb9f1470349b3be140e7b70811e7b received from {username='slave'} at 127.26.25.135:48771
I20240502 14:00:54.129745 3039 tablet_copy_service.cc:433] P 452ecbc085e54db49c3b01c20a96f860: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-c47fb9f1470349b3be140e7b70811e7b on tablet c47fb9f1470349b3be140e7b70811e7b with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:54.134740 5902 ts_tablet_manager.cc:902] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:54.135514 5902 tablet_copy_client.cc:308] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.136:43131
I20240502 14:00:54.136255 3304 tablet_copy_service.cc:140] P da3f8160557e454f8547f45e80c2f013: Received BeginTabletCopySession request for tablet 6346d5ea09834bddb483da26f9bce936 from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:51927)
I20240502 14:00:54.136387 3304 tablet_copy_service.cc:161] P da3f8160557e454f8547f45e80c2f013: Beginning new tablet copy session on tablet 6346d5ea09834bddb483da26f9bce936 from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:51927: session id = 478dfa53d7944f38b4e6cabee21b2d01-6346d5ea09834bddb483da26f9bce936
I20240502 14:00:54.137706 3304 tablet_copy_source_session.cc:205] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.139691 5902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6346d5ea09834bddb483da26f9bce936. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.142001 5902 tablet_copy_client.cc:783] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.142256 5902 tablet_copy_client.cc:647] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.142835 5907 ts_tablet_manager.cc:902] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:54.143630 5907 tablet_copy_client.cc:308] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.136:43131
I20240502 14:00:54.144133 3303 tablet_copy_service.cc:140] P da3f8160557e454f8547f45e80c2f013: Received BeginTabletCopySession request for tablet b985ba4b9f534827919cbb24daa4de0e from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:51927)
I20240502 14:00:54.144238 3303 tablet_copy_service.cc:161] P da3f8160557e454f8547f45e80c2f013: Beginning new tablet copy session on tablet b985ba4b9f534827919cbb24daa4de0e from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:51927: session id = 478dfa53d7944f38b4e6cabee21b2d01-b985ba4b9f534827919cbb24daa4de0e
I20240502 14:00:54.145195 3303 tablet_copy_source_session.cc:205] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.146757 5902 tablet_copy_client.cc:523] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.150293 5902 tablet_bootstrap.cc:492] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:54.152271 5890 tablet_bootstrap.cc:492] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.152652 5890 tablet_bootstrap.cc:492] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860: Bootstrap complete.
I20240502 14:00:54.152925 5907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b985ba4b9f534827919cbb24daa4de0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.152948 5890 ts_tablet_manager.cc:1367] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.031s user 0.026s sys 0.004s
I20240502 14:00:54.153537 5890 raft_consensus.cc:348] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:54.153913 5890 raft_consensus.cc:729] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: LEARNER
I20240502 14:00:54.154099 5890 consensus_queue.cc:260] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:54.154464 5890 ts_tablet_manager.cc:1398] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:54.155671 694 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 452ecbc085e54db49c3b01c20a96f860-95101f5c218746eab5095b984181a5fc received from {username='slave'} at 127.26.25.134:41957
I20240502 14:00:54.155758 694 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 452ecbc085e54db49c3b01c20a96f860-95101f5c218746eab5095b984181a5fc on tablet 95101f5c218746eab5095b984181a5fc with peer 452ecbc085e54db49c3b01c20a96f860
I20240502 14:00:54.158535 5902 tablet_bootstrap.cc:492] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.158849 5902 tablet_bootstrap.cc:492] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:00:54.159035 5902 ts_tablet_manager.cc:1367] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.003s
I20240502 14:00:54.159540 5902 raft_consensus.cc:348] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:54.159689 5902 raft_consensus.cc:729] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:00:54.159767 5902 consensus_queue.cc:260] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:54.160069 5902 ts_tablet_manager.cc:1398] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 14:00:54.160894 3304 tablet_copy_service.cc:342] P da3f8160557e454f8547f45e80c2f013: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-6346d5ea09834bddb483da26f9bce936 received from {username='slave'} at 127.26.25.132:51927
I20240502 14:00:54.160985 5907 tablet_copy_client.cc:783] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.161000 3304 tablet_copy_service.cc:433] P da3f8160557e454f8547f45e80c2f013: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-6346d5ea09834bddb483da26f9bce936 on tablet 6346d5ea09834bddb483da26f9bce936 with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:00:54.161137 5907 tablet_copy_client.cc:647] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.162585 5907 tablet_copy_client.cc:523] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.163865 5907 tablet_bootstrap.cc:492] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:54.167517 5907 tablet_bootstrap.cc:492] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.167749 5907 tablet_bootstrap.cc:492] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:00:54.167940 5907 ts_tablet_manager.cc:1367] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.005s sys 0.000s
I20240502 14:00:54.168337 5907 raft_consensus.cc:348] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:54.168442 5907 raft_consensus.cc:729] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:00:54.168550 5907 consensus_queue.cc:260] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:54.168843 5907 ts_tablet_manager.cc:1398] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:54.169325 3304 tablet_copy_service.cc:342] P da3f8160557e454f8547f45e80c2f013: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-b985ba4b9f534827919cbb24daa4de0e received from {username='slave'} at 127.26.25.132:51927
I20240502 14:00:54.169405 3304 tablet_copy_service.cc:433] P da3f8160557e454f8547f45e80c2f013: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-b985ba4b9f534827919cbb24daa4de0e on tablet b985ba4b9f534827919cbb24daa4de0e with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:00:54.176940 5836 ts_tablet_manager.cc:902] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:54.177584 5836 tablet_copy_client.cc:308] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:00:54.178082 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 4aa562f43a154941a6e645e04757870d from peer 58107b611f8f475a910a6fd54dfd1fb9 ({username='slave'} at 127.26.25.131:52409)
I20240502 14:00:54.178191 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 4aa562f43a154941a6e645e04757870d from peer 58107b611f8f475a910a6fd54dfd1fb9 at {username='slave'} at 127.26.25.131:52409: session id = 58107b611f8f475a910a6fd54dfd1fb9-4aa562f43a154941a6e645e04757870d
I20240502 14:00:54.178992 2906 tablet_copy_source_session.cc:205] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.179940 5836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aa562f43a154941a6e645e04757870d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.182257 5836 tablet_copy_client.cc:783] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.182432 5836 tablet_copy_client.cc:647] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.183917 5836 tablet_copy_client.cc:523] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.185173 5836 tablet_bootstrap.cc:492] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:54.188783 5836 tablet_bootstrap.cc:492] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.189052 5836 tablet_bootstrap.cc:492] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap complete.
I20240502 14:00:54.189226 5836 ts_tablet_manager.cc:1367] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240502 14:00:54.189714 5836 raft_consensus.cc:348] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:54.189849 5836 raft_consensus.cc:729] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: LEARNER
I20240502 14:00:54.189970 5836 consensus_queue.cc:260] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:54.190300 5836 ts_tablet_manager.cc:1398] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:54.190761 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-4aa562f43a154941a6e645e04757870d received from {username='slave'} at 127.26.25.131:52409
I20240502 14:00:54.190840 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-4aa562f43a154941a6e645e04757870d on tablet 4aa562f43a154941a6e645e04757870d with peer 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:00:54.227880 5851 ts_tablet_manager.cc:902] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:54.228392 5851 tablet_copy_client.cc:308] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:54.228881 694 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 5f321621da5d481b8d1d2b25ddf4b395 from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:37651)
I20240502 14:00:54.228989 694 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 5f321621da5d481b8d1d2b25ddf4b395 from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:37651: session id = 2b6796cf0e2e496e9dd641beb432f485-5f321621da5d481b8d1d2b25ddf4b395
I20240502 14:00:54.229743 694 tablet_copy_source_session.cc:205] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.230551 5851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f321621da5d481b8d1d2b25ddf4b395. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.232462 5851 tablet_copy_client.cc:783] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.232618 5851 tablet_copy_client.cc:647] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.233863 5851 tablet_copy_client.cc:523] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.234932 5851 tablet_bootstrap.cc:492] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:54.259478 5907 ts_tablet_manager.cc:902] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:54.260625 5907 tablet_copy_client.cc:308] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:00:54.261119 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet 3bcaf05a01194e919efca9cc08d61c8f from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:57613)
I20240502 14:00:54.261240 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet 3bcaf05a01194e919efca9cc08d61c8f from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:57613: session id = 478dfa53d7944f38b4e6cabee21b2d01-3bcaf05a01194e919efca9cc08d61c8f
I20240502 14:00:54.262290 3172 tablet_copy_source_session.cc:205] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.262616 5851 tablet_bootstrap.cc:492] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.263543 5851 tablet_bootstrap.cc:492] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:54.263622 5907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bcaf05a01194e919efca9cc08d61c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.263744 5851 ts_tablet_manager.cc:1367] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.029s user 0.015s sys 0.011s
I20240502 14:00:54.264293 5851 raft_consensus.cc:348] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:54.264406 5851 raft_consensus.cc:729] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:54.264501 5851 consensus_queue.cc:260] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:54.264822 5851 ts_tablet_manager.cc:1398] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:54.266243 5907 tablet_copy_client.cc:783] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.266415 5907 tablet_copy_client.cc:647] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.267006 694 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-5f321621da5d481b8d1d2b25ddf4b395 received from {username='slave'} at 127.26.25.135:37651
I20240502 14:00:54.267108 694 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-5f321621da5d481b8d1d2b25ddf4b395 on tablet 5f321621da5d481b8d1d2b25ddf4b395 with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:54.267423 5907 tablet_copy_client.cc:523] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.268715 5907 tablet_bootstrap.cc:492] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:54.272522 5907 tablet_bootstrap.cc:492] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.272850 5907 tablet_bootstrap.cc:492] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:00:54.273070 5907 ts_tablet_manager.cc:1367] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.001s
I20240502 14:00:54.273566 5907 raft_consensus.cc:348] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:54.273684 5907 raft_consensus.cc:729] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:00:54.273811 5907 consensus_queue.cc:260] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:54.274124 5907 ts_tablet_manager.cc:1398] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:54.275189 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-3bcaf05a01194e919efca9cc08d61c8f received from {username='slave'} at 127.26.25.132:57613
I20240502 14:00:54.275277 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-3bcaf05a01194e919efca9cc08d61c8f on tablet 3bcaf05a01194e919efca9cc08d61c8f with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:00:54.288556 5851 ts_tablet_manager.cc:902] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:54.289047 5851 tablet_copy_client.cc:308] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:44715
I20240502 14:00:54.290179 430 tablet_copy_service.cc:140] P f003b732dd0f46958262b42f8d20305e: Received BeginTabletCopySession request for tablet ce81659e36874bb89da4d2268eb40aea from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:55895)
I20240502 14:00:54.290330 430 tablet_copy_service.cc:161] P f003b732dd0f46958262b42f8d20305e: Beginning new tablet copy session on tablet ce81659e36874bb89da4d2268eb40aea from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:55895: session id = 2b6796cf0e2e496e9dd641beb432f485-ce81659e36874bb89da4d2268eb40aea
I20240502 14:00:54.291800 430 tablet_copy_source_session.cc:205] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.296840 5851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce81659e36874bb89da4d2268eb40aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.299278 5851 tablet_copy_client.cc:783] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.299477 5851 tablet_copy_client.cc:647] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.302304 5851 tablet_copy_client.cc:523] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.304250 5851 tablet_bootstrap.cc:492] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:54.310767 5959 ts_tablet_manager.cc:902] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:54.311489 5959 tablet_copy_client.cc:308] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35251
I20240502 14:00:54.311967 562 tablet_copy_service.cc:140] P c351a89a5fe34e788b9d8064fbffa751: Received BeginTabletCopySession request for tablet fee3c05d88514648b8291e01eda2db2b from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:53809)
I20240502 14:00:54.312086 562 tablet_copy_service.cc:161] P c351a89a5fe34e788b9d8064fbffa751: Beginning new tablet copy session on tablet fee3c05d88514648b8291e01eda2db2b from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:53809: session id = 2b6796cf0e2e496e9dd641beb432f485-fee3c05d88514648b8291e01eda2db2b
I20240502 14:00:54.313146 562 tablet_copy_source_session.cc:205] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.314184 5959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fee3c05d88514648b8291e01eda2db2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.316790 5959 tablet_copy_client.cc:783] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.316977 5959 tablet_copy_client.cc:647] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.319401 5959 tablet_copy_client.cc:523] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.322214 5959 tablet_bootstrap.cc:492] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:54.326421 5959 tablet_bootstrap.cc:492] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.326784 5959 tablet_bootstrap.cc:492] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:54.327010 5959 ts_tablet_manager.cc:1367] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 14:00:54.327546 5959 raft_consensus.cc:348] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:54.327880 5959 raft_consensus.cc:729] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:54.328099 5959 consensus_queue.cc:260] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:54.328544 5959 ts_tablet_manager.cc:1398] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:54.331240 562 tablet_copy_service.cc:342] P c351a89a5fe34e788b9d8064fbffa751: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-fee3c05d88514648b8291e01eda2db2b received from {username='slave'} at 127.26.25.135:53809
I20240502 14:00:54.331331 562 tablet_copy_service.cc:433] P c351a89a5fe34e788b9d8064fbffa751: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-fee3c05d88514648b8291e01eda2db2b on tablet fee3c05d88514648b8291e01eda2db2b with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:54.336604 5851 tablet_bootstrap.cc:492] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.336931 5851 tablet_bootstrap.cc:492] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:54.337128 5851 ts_tablet_manager.cc:1367] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.033s user 0.018s sys 0.008s
I20240502 14:00:54.337965 5851 raft_consensus.cc:348] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:54.338109 5851 raft_consensus.cc:729] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:54.338201 5851 consensus_queue.cc:260] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:54.338469 5851 ts_tablet_manager.cc:1398] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:54.339112 430 tablet_copy_service.cc:342] P f003b732dd0f46958262b42f8d20305e: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-ce81659e36874bb89da4d2268eb40aea received from {username='slave'} at 127.26.25.135:55895
I20240502 14:00:54.339210 430 tablet_copy_service.cc:433] P f003b732dd0f46958262b42f8d20305e: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-ce81659e36874bb89da4d2268eb40aea on tablet ce81659e36874bb89da4d2268eb40aea with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:54.346808 5980 ts_tablet_manager.cc:902] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751: Initiating tablet copy from peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:54.347765 5980 tablet_copy_client.cc:308] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.132:33443
I20240502 14:00:54.348489 826 tablet_copy_service.cc:140] P 478dfa53d7944f38b4e6cabee21b2d01: Received BeginTabletCopySession request for tablet 56a192eba0af4486a212972ffdfacf9d from peer c351a89a5fe34e788b9d8064fbffa751 ({username='slave'} at 127.26.25.130:33155)
I20240502 14:00:54.348593 826 tablet_copy_service.cc:161] P 478dfa53d7944f38b4e6cabee21b2d01: Beginning new tablet copy session on tablet 56a192eba0af4486a212972ffdfacf9d from peer c351a89a5fe34e788b9d8064fbffa751 at {username='slave'} at 127.26.25.130:33155: session id = c351a89a5fe34e788b9d8064fbffa751-56a192eba0af4486a212972ffdfacf9d
I20240502 14:00:54.349726 826 tablet_copy_source_session.cc:205] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.351043 5980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56a192eba0af4486a212972ffdfacf9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.353506 5980 tablet_copy_client.cc:783] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.353713 5980 tablet_copy_client.cc:647] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.355237 5980 tablet_copy_client.cc:523] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.356585 5880 ts_tablet_manager.cc:902] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:54.357054 5880 tablet_copy_client.cc:308] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:54.357626 5980 tablet_bootstrap.cc:492] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:54.360651 694 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet a658bde2687344beabf4d8a0ff8988e2 from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:41615)
I20240502 14:00:54.360772 694 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet a658bde2687344beabf4d8a0ff8988e2 from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:41615: session id = f003b732dd0f46958262b42f8d20305e-a658bde2687344beabf4d8a0ff8988e2
I20240502 14:00:54.361774 694 tablet_copy_source_session.cc:205] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.362774 5880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a658bde2687344beabf4d8a0ff8988e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.364446 5980 tablet_bootstrap.cc:492] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.364727 5980 tablet_bootstrap.cc:492] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751: Bootstrap complete.
I20240502 14:00:54.364982 5880 tablet_copy_client.cc:783] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.365145 5880 tablet_copy_client.cc:647] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.365687 5980 ts_tablet_manager.cc:1367] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.002s
I20240502 14:00:54.366575 5980 raft_consensus.cc:348] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:54.366710 5980 raft_consensus.cc:729] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: LEARNER
I20240502 14:00:54.366806 5980 consensus_queue.cc:260] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:54.367071 5980 ts_tablet_manager.cc:1398] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:54.367193 5880 tablet_copy_client.cc:523] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.369104 826 tablet_copy_service.cc:342] P 478dfa53d7944f38b4e6cabee21b2d01: Request end of tablet copy session c351a89a5fe34e788b9d8064fbffa751-56a192eba0af4486a212972ffdfacf9d received from {username='slave'} at 127.26.25.130:33155
I20240502 14:00:54.369174 826 tablet_copy_service.cc:433] P 478dfa53d7944f38b4e6cabee21b2d01: ending tablet copy session c351a89a5fe34e788b9d8064fbffa751-56a192eba0af4486a212972ffdfacf9d on tablet 56a192eba0af4486a212972ffdfacf9d with peer c351a89a5fe34e788b9d8064fbffa751
I20240502 14:00:54.370457 5868 ts_tablet_manager.cc:902] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:54.370918 5868 tablet_copy_client.cc:308] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.134:39061
I20240502 14:00:54.371393 3039 tablet_copy_service.cc:140] P 452ecbc085e54db49c3b01c20a96f860: Received BeginTabletCopySession request for tablet 923a5d2ccd0448b381d88ced69ae8fe2 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:42755)
I20240502 14:00:54.371510 3039 tablet_copy_service.cc:161] P 452ecbc085e54db49c3b01c20a96f860: Beginning new tablet copy session on tablet 923a5d2ccd0448b381d88ced69ae8fe2 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:42755: session id = da3f8160557e454f8547f45e80c2f013-923a5d2ccd0448b381d88ced69ae8fe2
I20240502 14:00:54.372013 5880 tablet_bootstrap.cc:492] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:54.372507 3039 tablet_copy_source_session.cc:205] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.373457 5868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923a5d2ccd0448b381d88ced69ae8fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.376295 5868 tablet_copy_client.cc:783] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.376479 5868 tablet_copy_client.cc:647] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.378654 5868 tablet_copy_client.cc:523] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.379251 5880 tablet_bootstrap.cc:492] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.379531 5880 tablet_bootstrap.cc:492] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:00:54.379722 5880 ts_tablet_manager.cc:1367] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20240502 14:00:54.380271 5868 tablet_bootstrap.cc:492] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:54.380299 5880 raft_consensus.cc:348] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:54.380436 5880 raft_consensus.cc:729] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:00:54.380548 5880 consensus_queue.cc:260] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:54.380869 5880 ts_tablet_manager.cc:1398] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:54.381392 694 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-a658bde2687344beabf4d8a0ff8988e2 received from {username='slave'} at 127.26.25.129:41615
I20240502 14:00:54.381474 694 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session f003b732dd0f46958262b42f8d20305e-a658bde2687344beabf4d8a0ff8988e2 on tablet a658bde2687344beabf4d8a0ff8988e2 with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:00:54.386049 5868 tablet_bootstrap.cc:492] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.386292 5868 tablet_bootstrap.cc:492] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:00:54.386430 5868 ts_tablet_manager.cc:1367] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 14:00:54.386893 5868 raft_consensus.cc:348] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:54.386986 5868 raft_consensus.cc:729] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:00:54.387053 5868 consensus_queue.cc:260] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:54.387346 5868 ts_tablet_manager.cc:1398] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:54.387976 3039 tablet_copy_service.cc:342] P 452ecbc085e54db49c3b01c20a96f860: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-923a5d2ccd0448b381d88ced69ae8fe2 received from {username='slave'} at 127.26.25.136:42755
I20240502 14:00:54.388062 3039 tablet_copy_service.cc:433] P 452ecbc085e54db49c3b01c20a96f860: ending tablet copy session da3f8160557e454f8547f45e80c2f013-923a5d2ccd0448b381d88ced69ae8fe2 on tablet 923a5d2ccd0448b381d88ced69ae8fe2 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:00:54.440047 3282 raft_consensus.cc:1203] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.443724 5880 ts_tablet_manager.cc:902] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:54.444535 5880 tablet_copy_client.cc:308] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.136:43131
I20240502 14:00:54.445207 3304 tablet_copy_service.cc:140] P da3f8160557e454f8547f45e80c2f013: Received BeginTabletCopySession request for tablet 094aa25b08b8498f9ae4493cafd289e3 from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:57497)
I20240502 14:00:54.445353 3304 tablet_copy_service.cc:161] P da3f8160557e454f8547f45e80c2f013: Beginning new tablet copy session on tablet 094aa25b08b8498f9ae4493cafd289e3 from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:57497: session id = f003b732dd0f46958262b42f8d20305e-094aa25b08b8498f9ae4493cafd289e3
I20240502 14:00:54.446377 3304 tablet_copy_source_session.cc:205] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:54.447796 5880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 094aa25b08b8498f9ae4493cafd289e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:54.450187 5880 tablet_copy_client.cc:783] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:54.450345 5880 tablet_copy_client.cc:647] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:54.451526 5880 tablet_copy_client.cc:523] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:54.452989 5880 tablet_bootstrap.cc:492] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:54.457134 5880 tablet_bootstrap.cc:492] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:54.457444 5880 tablet_bootstrap.cc:492] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:00:54.457614 5880 ts_tablet_manager.cc:1367] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20240502 14:00:54.458006 5880 raft_consensus.cc:348] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:54.458112 5880 raft_consensus.cc:729] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:00:54.458216 5880 consensus_queue.cc:260] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:54.458576 5880 ts_tablet_manager.cc:1398] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 14:00:54.459087 3304 tablet_copy_service.cc:342] P da3f8160557e454f8547f45e80c2f013: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-094aa25b08b8498f9ae4493cafd289e3 received from {username='slave'} at 127.26.25.129:57497
I20240502 14:00:54.459177 3304 tablet_copy_service.cc:433] P da3f8160557e454f8547f45e80c2f013: ending tablet copy session f003b732dd0f46958262b42f8d20305e-094aa25b08b8498f9ae4493cafd289e3 on tablet 094aa25b08b8498f9ae4493cafd289e3 with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:00:54.460883 2884 raft_consensus.cc:1203] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.530969 3151 raft_consensus.cc:1203] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:54.538663 405 raft_consensus.cc:1203] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.554500 805 raft_consensus.cc:1203] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.560235 3282 raft_consensus.cc:1203] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.575078 3019 raft_consensus.cc:1203] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:54.580530 805 raft_consensus.cc:1203] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.590281 674 raft_consensus.cc:1203] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.610260 3151 raft_consensus.cc:1203] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.625059 3151 raft_consensus.cc:1203] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:54.632707 405 raft_consensus.cc:1203] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.759800 674 raft_consensus.cc:1203] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.762460 542 raft_consensus.cc:1203] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.802784 805 raft_consensus.cc:1203] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.809461 3282 raft_consensus.cc:1203] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.811643 3150 raft_consensus.cc:1203] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:54.887873 405 raft_consensus.cc:1203] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.894698 3150 raft_consensus.cc:1203] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.952524 405 raft_consensus.cc:1203] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:54.957612 6064 raft_consensus.cc:1050] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01: attempting to promote NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 to VOTER
I20240502 14:00:54.958039 6064 consensus_queue.cc:237] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:00:54.959272 2886 raft_consensus.cc:1261] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:54.959391 3282 raft_consensus.cc:1261] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:54.959436 541 raft_consensus.cc:1261] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:54.959745 6174 consensus_queue.cc:1035] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:54.959995 6064 consensus_queue.cc:1035] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:54.960203 5429 consensus_queue.cc:1035] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:54.962567 6064 raft_consensus.cc:2932] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:54.962955 541 raft_consensus.cc:2932] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:54.963563 2886 raft_consensus.cc:2932] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:54.965361 3282 raft_consensus.cc:2932] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:54.968144 32732 catalog_manager.cc:5484] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 2 to 3, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:54.971683 805 consensus_queue.cc:237] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:00:54.972985 2886 raft_consensus.cc:1261] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:54.972988 541 raft_consensus.cc:1261] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:54.973372 5429 consensus_queue.cc:1035] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:54.973578 5582 consensus_queue.cc:1035] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:54.975144 5429 raft_consensus.cc:2932] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:54.975626 541 raft_consensus.cc:2932] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:54.976044 2886 raft_consensus.cc:2932] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:54.976647 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 72395d91ff48409693431748b3efeb6d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:54.978195 32726 catalog_manager.cc:5484] T 72395d91ff48409693431748b3efeb6d P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:54.981374 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet 72395d91ff48409693431748b3efeb6d with delete_type TABLET_DATA_TOMBSTONED (TS da3f8160557e454f8547f45e80c2f013 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:54.981732 6182 tablet_replica.cc:330] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:54.981853 6182 raft_consensus.cc:2229] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:54.981978 6182 raft_consensus.cc:2258] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:54.982547 6182 ts_tablet_manager.cc:1875] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:54.984828 6182 ts_tablet_manager.cc:1888] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:54.984908 6182 log.cc:1196] T 72395d91ff48409693431748b3efeb6d P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/72395d91ff48409693431748b3efeb6d
I20240502 14:00:54.985564 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 72395d91ff48409693431748b3efeb6d (table rebalancer_extra_table_24 [id=7157a49edcbc4ab4a4b4cb35ef78edac]) successfully deleted
I20240502 14:00:54.985797 5380 raft_consensus.cc:1050] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:00:54.986335 5380 consensus_queue.cc:237] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:54.987704 2886 raft_consensus.cc:1261] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:54.987704 3018 raft_consensus.cc:1261] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:54.988138 5597 consensus_queue.cc:1035] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:54.988296 6184 consensus_queue.cc:1035] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:54.988551 405 raft_consensus.cc:1261] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:54.988943 6184 consensus_queue.cc:1035] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:54.990593 6184 raft_consensus.cc:2932] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:54.991122 405 raft_consensus.cc:2932] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:54.991189 3018 raft_consensus.cc:2932] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:54.994544 32729 catalog_manager.cc:5484] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:54.995265 2886 raft_consensus.cc:2932] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:54.997859 3150 consensus_queue.cc:237] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:54.999130 405 raft_consensus.cc:1261] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:54.999527 6184 consensus_queue.cc:1035] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.000027 2886 raft_consensus.cc:1261] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.000892 6184 consensus_queue.cc:1035] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.001888 6066 raft_consensus.cc:2932] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.002843 2886 raft_consensus.cc:2932] T 4fc8d5ad25334f0a8c2efd17b5779330 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.003088 5104 raft_consensus.cc:1050] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:00:55.003576 5104 consensus_queue.cc:237] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:55.003710 405 raft_consensus.cc:2932] T 4fc8d5ad25334f0a8c2efd17b5779330 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.005203 3018 raft_consensus.cc:1261] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.005399 805 raft_consensus.cc:1261] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.005520 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4fc8d5ad25334f0a8c2efd17b5779330 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.005682 5202 consensus_queue.cc:1035] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.006691 405 raft_consensus.cc:1261] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.006915 32726 catalog_manager.cc:5484] T 4fc8d5ad25334f0a8c2efd17b5779330 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.007277 5202 consensus_queue.cc:1035] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.007596 5202 consensus_queue.cc:1035] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.010107 5202 raft_consensus.cc:2932] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.011961 409 raft_consensus.cc:2932] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.013569 32732 catalog_manager.cc:5484] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.014489 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fc8d5ad25334f0a8c2efd17b5779330 with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:55.020872 6193 tablet_replica.cc:330] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:55.021070 6193 raft_consensus.cc:2229] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.021235 6193 raft_consensus.cc:2258] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.021831 6193 ts_tablet_manager.cc:1875] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.024267 6193 ts_tablet_manager.cc:1888] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.024376 6193 log.cc:1196] T 4fc8d5ad25334f0a8c2efd17b5779330 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/4fc8d5ad25334f0a8c2efd17b5779330
I20240502 14:00:55.024951 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 4fc8d5ad25334f0a8c2efd17b5779330 (table rebalancer_extra_table_32 [id=274e7365daae4e2dba024f22f983d794]) successfully deleted
I20240502 14:00:55.028648 805 raft_consensus.cc:2932] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.031097 3282 consensus_queue.cc:237] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:55.031661 5655 raft_consensus.cc:1050] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751: attempting to promote NON_VOTER da3f8160557e454f8547f45e80c2f013 to VOTER
I20240502 14:00:55.032224 5655 consensus_queue.cc:237] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:55.032548 805 raft_consensus.cc:1261] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.032960 5104 consensus_queue.cc:1035] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.034056 2886 raft_consensus.cc:1261] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.034205 3282 raft_consensus.cc:1261] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.034713 3019 raft_consensus.cc:1261] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.034833 6068 raft_consensus.cc:1050] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:00:55.035308 6068 consensus_queue.cc:237] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:55.036844 674 raft_consensus.cc:1261] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.036888 2886 raft_consensus.cc:1261] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.036986 6051 consensus_queue.cc:1035] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.037365 6051 consensus_queue.cc:1035] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.037716 5282 consensus_queue.cc:1035] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.038587 5104 raft_consensus.cc:2932] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.040721 32719 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b985ba4b9f534827919cbb24daa4de0e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.041152 805 raft_consensus.cc:2932] T b985ba4b9f534827919cbb24daa4de0e P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.041277 5282 consensus_queue.cc:1035] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.042850 32729 catalog_manager.cc:5484] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.043157 5358 raft_consensus.cc:1050] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER da3f8160557e454f8547f45e80c2f013 to VOTER
I20240502 14:00:55.043642 5358 consensus_queue.cc:237] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:55.045133 5098 raft_consensus.cc:1050] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:00:55.045679 5098 consensus_queue.cc:237] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:55.047345 3019 raft_consensus.cc:1261] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.047899 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet b985ba4b9f534827919cbb24daa4de0e with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:55.048072 3283 raft_consensus.cc:1261] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.048543 409 raft_consensus.cc:1261] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.048877 6052 consensus_queue.cc:1035] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.049047 6202 consensus_queue.cc:1035] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.049477 6134 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.049899 6134 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.050689 541 raft_consensus.cc:1261] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.051081 5711 consensus_queue.cc:1035] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.051601 3151 raft_consensus.cc:1261] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.051934 408 raft_consensus.cc:1261] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.054415 5711 consensus_queue.cc:1035] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.057969 6052 raft_consensus.cc:2932] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.058341 5202 consensus_queue.cc:1035] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.060606 406 raft_consensus.cc:2932] T b985ba4b9f534827919cbb24daa4de0e P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.062870 5711 raft_consensus.cc:1050] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER 452ecbc085e54db49c3b01c20a96f860 to VOTER
I20240502 14:00:55.063274 5711 consensus_queue.cc:237] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:00:55.063622 6065 raft_consensus.cc:1050] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:00:55.064042 6065 consensus_queue.cc:237] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:55.064563 32729 catalog_manager.cc:5484] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.066298 800 raft_consensus.cc:1261] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.066962 805 raft_consensus.cc:1261] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.067260 3018 raft_consensus.cc:2932] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.068372 542 raft_consensus.cc:1261] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.068825 3017 raft_consensus.cc:1261] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.069265 409 raft_consensus.cc:1261] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.069223 5282 raft_consensus.cc:2932] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.069525 405 raft_consensus.cc:1261] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.069976 6065 consensus_queue.cc:1035] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.070287 6065 consensus_queue.cc:1035] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.070546 6065 consensus_queue.cc:1035] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.070803 6065 consensus_queue.cc:1035] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.071841 408 raft_consensus.cc:2932] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.074151 542 consensus_queue.cc:237] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:55.074440 674 raft_consensus.cc:2932] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.076686 2886 raft_consensus.cc:2932] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.076723 32732 catalog_manager.cc:5484] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.077517 3017 raft_consensus.cc:1261] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.078681 5674 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.079185 3150 raft_consensus.cc:1261] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.079663 5202 raft_consensus.cc:1050] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:00:55.080145 5202 consensus_queue.cc:237] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:55.081880 3017 raft_consensus.cc:1261] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.082816 6193 tablet_replica.cc:330] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:55.082849 5104 consensus_queue.cc:1035] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.082979 6193 raft_consensus.cc:2229] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.083086 6193 pending_rounds.cc:62] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860: Trying to abort 1 pending ops.
I20240502 14:00:55.083144 6193 pending_rounds.cc:68] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7023241031693533184 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "b985ba4b9f534827919cbb24daa4de0e" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } } }
I20240502 14:00:55.083267 6193 raft_consensus.cc:2894] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20240502 14:00:55.083318 6193 raft_consensus.cc:2866] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 14:00:55.083411 6193 raft_consensus.cc:2258] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.083410 3283 raft_consensus.cc:2932] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.084909 3284 raft_consensus.cc:1261] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.086542 6193 ts_tablet_manager.cc:1875] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.087208 2886 raft_consensus.cc:1261] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.087906 6208 raft_consensus.cc:2932] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.089236 6052 consensus_queue.cc:1035] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.089577 6052 consensus_queue.cc:1035] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.089263 6193 ts_tablet_manager.cc:1888] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.089978 6193 log.cc:1196] T b985ba4b9f534827919cbb24daa4de0e P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/b985ba4b9f534827919cbb24daa4de0e
I20240502 14:00:55.089907 2884 raft_consensus.cc:2932] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.090611 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet b985ba4b9f534827919cbb24daa4de0e (table rebalancer_extra_table_38 [id=f203281184c04e388bc5589f549f4a64]) successfully deleted
I20240502 14:00:55.091130 5711 consensus_queue.cc:1035] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.091288 5098 consensus_queue.cc:1035] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.091261 540 raft_consensus.cc:2932] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.091485 406 raft_consensus.cc:2932] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.092444 5104 consensus_queue.cc:1035] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.092440 3150 raft_consensus.cc:2932] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.093789 3016 consensus_queue.cc:237] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:55.095552 32729 catalog_manager.cc:5484] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.096643 406 raft_consensus.cc:1261] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.096634 2886 raft_consensus.cc:1261] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.097077 5282 consensus_queue.cc:1035] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.097522 5282 consensus_queue.cc:1035] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.099210 5282 raft_consensus.cc:2932] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.099695 5655 raft_consensus.cc:2932] T 093babce464c4152ab6ade948dc7a616 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.101670 3016 raft_consensus.cc:2932] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.103312 406 raft_consensus.cc:2932] T 6423e05535aa41d5923f570c7b73f61c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.103916 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6423e05535aa41d5923f570c7b73f61c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.104092 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 093babce464c4152ab6ade948dc7a616 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.105156 3281 raft_consensus.cc:2932] T 093babce464c4152ab6ade948dc7a616 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.105281 32731 catalog_manager.cc:5484] T 6423e05535aa41d5923f570c7b73f61c P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.105592 32731 catalog_manager.cc:5484] T 093babce464c4152ab6ade948dc7a616 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.108992 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 6423e05535aa41d5923f570c7b73f61c with delete_type TABLET_DATA_TOMBSTONED (TS 58107b611f8f475a910a6fd54dfd1fb9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:55.109344 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 093babce464c4152ab6ade948dc7a616 with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:55.110148 6232 tablet_replica.cc:330] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:55.110364 6232 raft_consensus.cc:2229] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.110911 2886 raft_consensus.cc:2932] T 6423e05535aa41d5923f570c7b73f61c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.111290 5637 raft_consensus.cc:1050] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:00:55.111734 6232 raft_consensus.cc:2258] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.111691 5637 consensus_queue.cc:237] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:55.112425 6232 ts_tablet_manager.cc:1875] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.113943 2886 raft_consensus.cc:1261] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.114399 3150 raft_consensus.cc:1261] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.114461 6068 consensus_queue.cc:1035] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.114944 6068 consensus_queue.cc:1035] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.115420 6232 ts_tablet_manager.cc:1888] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.115542 6232 log.cc:1196] T 6423e05535aa41d5923f570c7b73f61c P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/6423e05535aa41d5923f570c7b73f61c
I20240502 14:00:55.115708 406 raft_consensus.cc:1261] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.116245 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 6423e05535aa41d5923f570c7b73f61c (table rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]) successfully deleted
I20240502 14:00:55.116291 6068 consensus_queue.cc:1035] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.122571 805 raft_consensus.cc:1261] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.123687 5098 consensus_queue.cc:1035] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.125366 805 raft_consensus.cc:1261] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.126741 6189 consensus_queue.cc:1035] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.129405 5104 raft_consensus.cc:2932] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.131453 801 raft_consensus.cc:2932] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.131683 6206 raft_consensus.cc:2932] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.132301 6208 raft_consensus.cc:2932] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.133316 3019 raft_consensus.cc:2932] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.134145 3281 raft_consensus.cc:2932] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.135057 541 raft_consensus.cc:2932] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.135890 32728 catalog_manager.cc:5484] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.137094 32731 catalog_manager.cc:5484] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.137991 3019 raft_consensus.cc:2932] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.139011 5098 raft_consensus.cc:2932] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:55.139261 32726 catalog_manager.cc:5484] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.139815 5637 raft_consensus.cc:2932] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.141160 2886 raft_consensus.cc:2932] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.141464 3018 raft_consensus.cc:2932] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:55.141950 405 raft_consensus.cc:2932] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.147054 32726 catalog_manager.cc:5484] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.148149 405 raft_consensus.cc:2932] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:55.148348 5674 raft_consensus.cc:1050] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 to VOTER
I20240502 14:00:55.148751 5674 consensus_queue.cc:237] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:00:55.150792 674 raft_consensus.cc:1261] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.150936 542 raft_consensus.cc:1261] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.151237 6233 tablet_replica.cc:330] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:55.151364 6233 raft_consensus.cc:2229] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.151494 6233 raft_consensus.cc:2258] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.151708 803 raft_consensus.cc:2932] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.153386 804 raft_consensus.cc:2932] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:55.154141 3151 raft_consensus.cc:2932] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.154718 3284 consensus_queue.cc:237] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:55.155169 32726 catalog_manager.cc:5484] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:55.157526 5582 raft_consensus.cc:1050] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01: attempting to promote NON_VOTER c351a89a5fe34e788b9d8064fbffa751 to VOTER
I20240502 14:00:55.157892 5582 consensus_queue.cc:237] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:55.158471 3018 raft_consensus.cc:1261] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.159193 405 raft_consensus.cc:1261] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.159169 5104 consensus_queue.cc:1035] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.159610 6206 consensus_queue.cc:1035] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.160449 804 raft_consensus.cc:1261] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.161224 5104 consensus_queue.cc:1035] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.161541 674 raft_consensus.cc:1261] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.162034 3018 raft_consensus.cc:1261] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.162055 6064 consensus_queue.cc:1035] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.162736 5104 raft_consensus.cc:2932] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.162976 542 raft_consensus.cc:1261] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.163313 6064 consensus_queue.cc:1035] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.163625 6064 consensus_queue.cc:1035] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.164170 672 consensus_queue.cc:237] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:00:55.165382 6233 ts_tablet_manager.cc:1875] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.165709 3018 raft_consensus.cc:1261] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:55.166051 6206 consensus_queue.cc:1035] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.166213 5358 consensus_queue.cc:1035] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.167304 409 raft_consensus.cc:1261] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:55.168157 5711 consensus_queue.cc:1035] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:55.168403 5711 consensus_queue.cc:1035] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:55.169380 32719 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6346d5ea09834bddb483da26f9bce936 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.170137 673 consensus_queue.cc:237] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:55.171079 3018 raft_consensus.cc:2932] T 6346d5ea09834bddb483da26f9bce936 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.171360 3151 raft_consensus.cc:1261] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:55.171746 5711 consensus_queue.cc:1035] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:55.172029 803 raft_consensus.cc:1261] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:55.172735 32728 catalog_manager.cc:5484] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.173336 6208 raft_consensus.cc:2932] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.174409 6221 raft_consensus.cc:2932] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:55.176406 6065 consensus_queue.cc:1035] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:55.177409 803 raft_consensus.cc:2932] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.177726 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5f321621da5d481b8d1d2b25ddf4b395 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.177878 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 95101f5c218746eab5095b984181a5fc with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.178349 804 raft_consensus.cc:2932] T 6346d5ea09834bddb483da26f9bce936 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.179714 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 6346d5ea09834bddb483da26f9bce936 with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:55.181205 32729 catalog_manager.cc:5484] T 5f321621da5d481b8d1d2b25ddf4b395 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.181425 3151 raft_consensus.cc:2932] T 5f321621da5d481b8d1d2b25ddf4b395 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.182830 6206 raft_consensus.cc:2932] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.183539 5429 raft_consensus.cc:2932] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:55.183997 405 raft_consensus.cc:2932] T 95101f5c218746eab5095b984181a5fc P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:55.184717 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f321621da5d481b8d1d2b25ddf4b395 with delete_type TABLET_DATA_TOMBSTONED (TS c351a89a5fe34e788b9d8064fbffa751 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:55.185348 540 raft_consensus.cc:2932] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.187677 6233 ts_tablet_manager.cc:1888] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.187753 6233 log.cc:1196] T 093babce464c4152ab6ade948dc7a616 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/093babce464c4152ab6ade948dc7a616
I20240502 14:00:55.188242 32732 catalog_manager.cc:5484] T 95101f5c218746eab5095b984181a5fc P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.188323 32731 catalog_manager.cc:5484] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.188470 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 093babce464c4152ab6ade948dc7a616 (table rebalancer_extra_table_54 [id=621920b4c989408eb3ab52ac1405f687]) successfully deleted
I20240502 14:00:55.188297 6233 tablet_replica.cc:330] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:55.189590 6233 raft_consensus.cc:2229] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.189703 6233 pending_rounds.cc:62] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5: Trying to abort 1 pending ops.
I20240502 14:00:55.189770 6233 pending_rounds.cc:68] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7023241032006987776 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "6346d5ea09834bddb483da26f9bce936" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } } }
I20240502 14:00:55.189913 6233 raft_consensus.cc:2894] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20240502 14:00:55.189985 6233 raft_consensus.cc:2866] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 14:00:55.190032 6233 raft_consensus.cc:2258] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.191550 672 raft_consensus.cc:2932] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.192823 6257 tablet_replica.cc:330] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:55.193409 6257 raft_consensus.cc:2229] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.193675 6257 raft_consensus.cc:2258] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.195187 3017 consensus_queue.cc:237] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:55.195762 3150 raft_consensus.cc:2932] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.196096 2886 consensus_queue.cc:237] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:00:55.196362 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 95101f5c218746eab5095b984181a5fc with delete_type TABLET_DATA_TOMBSTONED (TS 478dfa53d7944f38b4e6cabee21b2d01 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:55.197361 406 raft_consensus.cc:1261] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.197834 6134 consensus_queue.cc:1035] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.199244 406 raft_consensus.cc:2932] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.199262 672 raft_consensus.cc:2932] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:55.202454 3150 raft_consensus.cc:1261] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.203104 540 raft_consensus.cc:2932] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:55.203454 2886 raft_consensus.cc:1261] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.203927 6235 consensus_queue.cc:1035] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.204488 6235 consensus_queue.cc:1035] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.204594 6233 ts_tablet_manager.cc:1875] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.205447 32728 catalog_manager.cc:5484] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.207234 6258 tablet_replica.cc:330] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:55.207538 6258 raft_consensus.cc:2229] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.207724 6258 raft_consensus.cc:2258] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.207922 6202 raft_consensus.cc:2932] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.209201 5380 raft_consensus.cc:1050] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485: attempting to promote NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 to VOTER
I20240502 14:00:55.209668 5380 consensus_queue.cc:237] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:00:55.210402 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c47fb9f1470349b3be140e7b70811e7b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.211143 2884 raft_consensus.cc:2932] T c47fb9f1470349b3be140e7b70811e7b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.211398 803 raft_consensus.cc:1261] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.211535 32732 catalog_manager.cc:5484] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.211784 6184 consensus_queue.cc:1035] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.212641 3150 raft_consensus.cc:2932] T c47fb9f1470349b3be140e7b70811e7b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.213299 804 consensus_queue.cc:237] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:55.213847 2884 raft_consensus.cc:1261] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.214495 387 tablet_service.cc:1514] Processing DeleteTablet for tablet c47fb9f1470349b3be140e7b70811e7b with delete_type TABLET_DATA_TOMBSTONED (TS f003b732dd0f46958262b42f8d20305e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:55.215235 6184 consensus_queue.cc:1035] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.215431 6261 tablet_replica.cc:330] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:55.215579 6261 raft_consensus.cc:2229] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.216079 6261 raft_consensus.cc:2258] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.217346 6261 ts_tablet_manager.cc:1875] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.218353 674 raft_consensus.cc:1261] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.220146 6258 ts_tablet_manager.cc:1875] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.220367 540 raft_consensus.cc:1261] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.221225 674 raft_consensus.cc:1261] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.221326 672 raft_consensus.cc:1261] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.221987 6233 ts_tablet_manager.cc:1888] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.222219 6233 log.cc:1196] T 6346d5ea09834bddb483da26f9bce936 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/6346d5ea09834bddb483da26f9bce936
I20240502 14:00:55.223156 6184 consensus_queue.cc:1035] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.223232 3017 raft_consensus.cc:2932] T 95101f5c218746eab5095b984181a5fc P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:55.224417 6257 ts_tablet_manager.cc:1875] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.224470 3019 raft_consensus.cc:2932] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:55.222790 6134 consensus_queue.cc:1035] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.228564 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 6346d5ea09834bddb483da26f9bce936 (table rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]) successfully deleted
I20240502 14:00:55.229152 5429 consensus_queue.cc:1035] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.229413 6134 raft_consensus.cc:2932] T 4aa562f43a154941a6e645e04757870d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.230772 5582 consensus_queue.cc:1035] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.231511 6258 ts_tablet_manager.cc:1888] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:55.231802 405 raft_consensus.cc:2932] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.232959 803 raft_consensus.cc:2932] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.234052 5429 raft_consensus.cc:2932] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:55.234501 6258 log.cc:1196] T 95101f5c218746eab5095b984181a5fc P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/95101f5c218746eab5095b984181a5fc
I20240502 14:00:55.234722 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4aa562f43a154941a6e645e04757870d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.235095 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 95101f5c218746eab5095b984181a5fc (table rebalance_test_table_3 [id=673d46c73d60440fb2f914f93a51b362]) successfully deleted
I20240502 14:00:55.235724 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 56a192eba0af4486a212972ffdfacf9d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.236768 673 raft_consensus.cc:2932] T 4aa562f43a154941a6e645e04757870d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.219789 6261 ts_tablet_manager.cc:1888] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.239259 6261 log.cc:1196] T c47fb9f1470349b3be140e7b70811e7b P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/c47fb9f1470349b3be140e7b70811e7b
I20240502 14:00:55.240757 32726 catalog_manager.cc:5484] T 4aa562f43a154941a6e645e04757870d P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.241060 6257 ts_tablet_manager.cc:1888] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:55.241145 6257 log.cc:1196] T 5f321621da5d481b8d1d2b25ddf4b395 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/5f321621da5d481b8d1d2b25ddf4b395
I20240502 14:00:55.242365 540 raft_consensus.cc:2932] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:55.245834 673 raft_consensus.cc:2932] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:55.248979 6184 raft_consensus.cc:1050] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:00:55.241688 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 5f321621da5d481b8d1d2b25ddf4b395 (table rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]) successfully deleted
I20240502 14:00:55.242352 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet c47fb9f1470349b3be140e7b70811e7b (table rebalancer_extra_table_64 [id=5d240759fa884da8bf576a9a2b0ef5e2]) successfully deleted
I20240502 14:00:55.249413 6184 consensus_queue.cc:237] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:55.250365 32728 catalog_manager.cc:5484] T 56a192eba0af4486a212972ffdfacf9d P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.251024 2886 raft_consensus.cc:1261] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.251423 6066 consensus_queue.cc:1035] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.252146 5597 raft_consensus.cc:2932] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.252756 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aa562f43a154941a6e645e04757870d with delete_type TABLET_DATA_TOMBSTONED (TS c351a89a5fe34e788b9d8064fbffa751 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:55.252980 6257 tablet_replica.cc:330] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:55.253119 6257 raft_consensus.cc:2229] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.253260 6257 raft_consensus.cc:2258] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.254356 6257 ts_tablet_manager.cc:1875] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.256790 2886 raft_consensus.cc:2932] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.257290 6257 ts_tablet_manager.cc:1888] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.257359 6257 log.cc:1196] T 4aa562f43a154941a6e645e04757870d P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/4aa562f43a154941a6e645e04757870d
I20240502 14:00:55.257880 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 4aa562f43a154941a6e645e04757870d (table rebalancer_extra_table_52 [id=d621bb0c4e764e628cf845d1ed77f115]) successfully deleted
I20240502 14:00:55.260293 805 raft_consensus.cc:2932] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.260735 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 56a192eba0af4486a212972ffdfacf9d with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:55.260985 6193 tablet_replica.cc:330] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:55.261145 6193 raft_consensus.cc:2229] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.261279 6193 raft_consensus.cc:2258] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.261989 803 raft_consensus.cc:1261] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.262049 6193 ts_tablet_manager.cc:1875] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.264029 6066 consensus_queue.cc:1035] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.264910 6193 ts_tablet_manager.cc:1888] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.264981 6193 log.cc:1196] T 56a192eba0af4486a212972ffdfacf9d P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/56a192eba0af4486a212972ffdfacf9d
I20240502 14:00:55.265507 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 56a192eba0af4486a212972ffdfacf9d (table rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]) successfully deleted
I20240502 14:00:55.266036 3019 raft_consensus.cc:1261] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.266639 673 raft_consensus.cc:2932] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.267158 5380 consensus_queue.cc:1035] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.267084 32732 catalog_manager.cc:5484] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.270906 5597 raft_consensus.cc:2932] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.272836 803 raft_consensus.cc:2932] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.277272 2884 raft_consensus.cc:2932] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.277917 3017 raft_consensus.cc:2932] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.279052 32728 catalog_manager.cc:5484] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.287304 3150 consensus_queue.cc:237] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:00:55.288611 2884 raft_consensus.cc:1261] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.288962 673 raft_consensus.cc:1261] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.289149 5597 consensus_queue.cc:1035] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.289686 5597 consensus_queue.cc:1035] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.290808 6066 raft_consensus.cc:2932] T bbae9f2a88db4a92b89cc12dfb837ad2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.290957 3150 consensus_queue.cc:237] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:55.292116 803 raft_consensus.cc:1261] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.292279 673 raft_consensus.cc:2932] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.292492 6184 consensus_queue.cc:1035] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.292668 3017 raft_consensus.cc:1261] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.293051 6184 consensus_queue.cc:1035] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.294286 6184 raft_consensus.cc:2932] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.294627 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bbae9f2a88db4a92b89cc12dfb837ad2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.295866 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3bcaf05a01194e919efca9cc08d61c8f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.296214 803 raft_consensus.cc:2932] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.296392 2884 raft_consensus.cc:2932] T bbae9f2a88db4a92b89cc12dfb837ad2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.298552 32728 catalog_manager.cc:5484] T bbae9f2a88db4a92b89cc12dfb837ad2 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:55.299237 6171 raft_consensus.cc:1050] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:00:55.299752 6171 consensus_queue.cc:237] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:55.302079 785 tablet_service.cc:1514] Processing DeleteTablet for tablet bbae9f2a88db4a92b89cc12dfb837ad2 with delete_type TABLET_DATA_TOMBSTONED (TS 478dfa53d7944f38b4e6cabee21b2d01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:55.302470 6258 tablet_replica.cc:330] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:55.302572 5202 raft_consensus.cc:1050] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:00:55.302626 6258 raft_consensus.cc:2229] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.302762 6258 raft_consensus.cc:2258] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.302954 5202 consensus_queue.cc:237] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:55.303573 6258 ts_tablet_manager.cc:1875] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.304376 406 raft_consensus.cc:1261] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.304585 3019 raft_consensus.cc:1261] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.304764 6189 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.304981 5104 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.305358 540 raft_consensus.cc:1261] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.305725 5715 consensus_queue.cc:1035] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.305828 6258 ts_tablet_manager.cc:1888] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.305892 6258 log.cc:1196] T bbae9f2a88db4a92b89cc12dfb837ad2 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/bbae9f2a88db4a92b89cc12dfb837ad2
I20240502 14:00:55.306512 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet bbae9f2a88db4a92b89cc12dfb837ad2 (table rebalancer_extra_table_10 [id=3efb8e76f34447f4adfad33ea46ac121]) successfully deleted
I20240502 14:00:55.306749 3149 raft_consensus.cc:1261] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.307147 5715 consensus_queue.cc:1035] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.307871 805 raft_consensus.cc:1261] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.308514 6067 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.309062 32728 catalog_manager.cc:5484] T 3bcaf05a01194e919efca9cc08d61c8f P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.310458 673 raft_consensus.cc:1261] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:55.311761 6280 consensus_queue.cc:1035] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:55.312052 6282 raft_consensus.cc:2932] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.313611 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bcaf05a01194e919efca9cc08d61c8f with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:55.313843 6233 tablet_replica.cc:330] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:55.314035 6233 raft_consensus.cc:2229] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.314190 6233 raft_consensus.cc:2258] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.314136 3017 raft_consensus.cc:2932] T 3bcaf05a01194e919efca9cc08d61c8f P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:55.315649 405 raft_consensus.cc:2932] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.315878 6233 ts_tablet_manager.cc:1875] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.318223 3017 raft_consensus.cc:2932] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.320204 805 raft_consensus.cc:2932] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.322012 32728 catalog_manager.cc:5484] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.324604 6233 ts_tablet_manager.cc:1888] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.324709 6233 log.cc:1196] T 3bcaf05a01194e919efca9cc08d61c8f P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/3bcaf05a01194e919efca9cc08d61c8f
I20240502 14:00:55.325559 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 3bcaf05a01194e919efca9cc08d61c8f (table rebalancer_extra_table_28 [id=8bbaf40cee36492e87629b1979f62041]) successfully deleted
I20240502 14:00:55.327760 6172 raft_consensus.cc:2932] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.328735 673 raft_consensus.cc:2932] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.329275 3150 raft_consensus.cc:2932] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.330998 542 raft_consensus.cc:2932] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.334399 32731 catalog_manager.cc:5484] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.338438 405 consensus_queue.cc:237] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:55.339716 3150 raft_consensus.cc:1261] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:55.339779 542 raft_consensus.cc:1261] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:55.340217 5715 consensus_queue.cc:1035] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:55.340600 6171 consensus_queue.cc:1035] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:55.342422 5715 raft_consensus.cc:2932] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.343078 542 raft_consensus.cc:2932] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.344283 3150 raft_consensus.cc:2932] T ce81659e36874bb89da4d2268eb40aea P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.345847 32720 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ce81659e36874bb89da4d2268eb40aea with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.350014 32734 catalog_manager.cc:5484] T ce81659e36874bb89da4d2268eb40aea P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.353410 653 tablet_service.cc:1514] Processing DeleteTablet for tablet ce81659e36874bb89da4d2268eb40aea with delete_type TABLET_DATA_TOMBSTONED (TS 58107b611f8f475a910a6fd54dfd1fb9 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:55.353680 6232 tablet_replica.cc:330] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:55.353845 6232 raft_consensus.cc:2229] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.354027 6232 raft_consensus.cc:2258] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.355152 6232 ts_tablet_manager.cc:1875] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.357306 6232 ts_tablet_manager.cc:1888] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:55.357375 6232 log.cc:1196] T ce81659e36874bb89da4d2268eb40aea P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/ce81659e36874bb89da4d2268eb40aea
I20240502 14:00:55.357905 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet ce81659e36874bb89da4d2268eb40aea (table rebalance_test_table_11 [id=441df53b66fd4ef7bb803ffdac4eb194]) successfully deleted
I20240502 14:00:55.369083 2884 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "909bc2eb63544db0b4b3a0764be7d4a1"
dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:51582
I20240502 14:00:55.369240 2884 raft_consensus.cc:595] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:55.391676 3284 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3"
dest_uuid: "da3f8160557e454f8547f45e80c2f013"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:45980
I20240502 14:00:55.391898 3284 raft_consensus.cc:595] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:55.402331 5655 raft_consensus.cc:1050] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:00:55.402815 5655 consensus_queue.cc:237] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:55.404414 3150 raft_consensus.cc:1261] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.404583 2884 raft_consensus.cc:1261] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.404994 673 raft_consensus.cc:1261] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.404994 6052 consensus_queue.cc:1035] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.405437 6052 consensus_queue.cc:1035] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.405776 6052 consensus_queue.cc:1035] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.407342 6235 raft_consensus.cc:1050] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: attempting to promote NON_VOTER da3f8160557e454f8547f45e80c2f013 to VOTER
I20240502 14:00:55.407744 6235 consensus_queue.cc:237] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:55.409256 3149 raft_consensus.cc:1261] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.409296 6200 raft_consensus.cc:2932] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.409585 2884 raft_consensus.cc:1261] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.409706 673 raft_consensus.cc:2932] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.410056 3284 raft_consensus.cc:1261] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.410073 6068 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.411346 6068 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.412205 6068 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.412328 2886 raft_consensus.cc:2932] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.414463 6068 raft_consensus.cc:2932] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.416574 2886 raft_consensus.cc:2932] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.416952 32734 catalog_manager.cc:5484] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.419193 3150 raft_consensus.cc:2932] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.419785 3281 raft_consensus.cc:2932] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.420790 32731 catalog_manager.cc:5484] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.425451 3149 raft_consensus.cc:2932] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:55.427336 540 consensus_queue.cc:237] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:55.429579 3150 raft_consensus.cc:1261] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.429961 6200 consensus_queue.cc:1035] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.430171 2886 raft_consensus.cc:1261] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.430653 6200 consensus_queue.cc:1035] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.432812 6052 raft_consensus.cc:2932] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.435240 3150 raft_consensus.cc:2932] T fee3c05d88514648b8291e01eda2db2b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.435438 2884 raft_consensus.cc:2932] T fee3c05d88514648b8291e01eda2db2b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.436776 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fee3c05d88514648b8291e01eda2db2b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.439198 32734 catalog_manager.cc:5484] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.442167 653 tablet_service.cc:1514] Processing DeleteTablet for tablet fee3c05d88514648b8291e01eda2db2b with delete_type TABLET_DATA_TOMBSTONED (TS 58107b611f8f475a910a6fd54dfd1fb9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:55.442404 6232 tablet_replica.cc:330] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:55.442574 6232 raft_consensus.cc:2229] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.442710 6232 raft_consensus.cc:2258] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.443342 6232 ts_tablet_manager.cc:1875] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.445477 6232 ts_tablet_manager.cc:1888] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.445649 6232 log.cc:1196] T fee3c05d88514648b8291e01eda2db2b P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/fee3c05d88514648b8291e01eda2db2b
I20240502 14:00:55.446216 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet fee3c05d88514648b8291e01eda2db2b (table rebalancer_extra_table_60 [id=4bef3bba6d9447d8a24755864aec22c2]) successfully deleted
I20240502 14:00:55.463953 3017 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "923a5d2ccd0448b381d88ced69ae8fe2"
dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38396
I20240502 14:00:55.464148 3017 raft_consensus.cc:595] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:55.488176 673 consensus_queue.cc:237] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:00:55.489498 803 raft_consensus.cc:1261] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.489776 540 raft_consensus.cc:1261] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.490044 6208 consensus_queue.cc:1035] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:00:55.490244 609 consensus_peers.cc:485] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): Couldn't send request to peer 4bcd7712aa844d98977a7d7e605c31f5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be795fbe277b4e9fac8561d6589b7c05. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.491600 6208 consensus_queue.cc:1035] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.493921 6221 raft_consensus.cc:2932] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:55.494452 540 raft_consensus.cc:2932] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:55.495536 803 raft_consensus.cc:2932] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:55.497861 32732 catalog_manager.cc:5484] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.510102 405 consensus_queue.cc:237] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:55.511961 540 raft_consensus.cc:1261] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.512097 673 raft_consensus.cc:1261] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:55.512380 341 consensus_peers.cc:485] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e -> Peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): Couldn't send request to peer 452ecbc085e54db49c3b01c20a96f860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1bdc7b131596496aa6779885fe5d9433. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.512604 6172 consensus_queue.cc:1035] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.512807 6171 consensus_queue.cc:1035] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.514927 6171 raft_consensus.cc:2932] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:55.515623 540 raft_consensus.cc:2932] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:55.515681 673 raft_consensus.cc:2932] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:55.518718 32728 catalog_manager.cc:5484] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.528600 673 consensus_queue.cc:237] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
W20240502 14:00:55.530182 609 consensus_peers.cc:485] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7113a6da4229422099dbd604a560fb95. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.530205 405 raft_consensus.cc:1261] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.530208 540 raft_consensus.cc:1261] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.530632 5098 consensus_queue.cc:1035] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.530838 6208 consensus_queue.cc:1035] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.532737 5711 raft_consensus.cc:2932] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.533162 540 raft_consensus.cc:2932] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.533236 405 raft_consensus.cc:2932] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.536850 32732 catalog_manager.cc:5484] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.546737 3017 consensus_queue.cc:237] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:55.548224 803 raft_consensus.cc:1261] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.548224 3281 raft_consensus.cc:1261] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:55.548519 2951 consensus_peers.cc:485] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 792bc86bad314a2bbf4bba3001c78894. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.548604 6202 consensus_queue.cc:1035] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.548765 6235 consensus_queue.cc:1035] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.550748 6202 raft_consensus.cc:2932] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:55.551091 3281 raft_consensus.cc:2932] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:55.554396 32731 catalog_manager.cc:5484] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.555620 803 raft_consensus.cc:2932] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:55.565688 673 consensus_queue.cc:237] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:55.567222 540 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.567412 803 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:55.567546 607 consensus_peers.cc:485] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5bed83e31be4129a19eb8c347514b80. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.567658 5098 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.568069 6208 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.569866 6208 raft_consensus.cc:2932] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:55.570312 540 raft_consensus.cc:2932] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:55.571035 803 raft_consensus.cc:2932] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:55.574782 32731 catalog_manager.cc:5484] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:55.578002 6208 raft_consensus.cc:1050] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:00:55.578444 6208 consensus_queue.cc:237] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:55.579931 405 raft_consensus.cc:1261] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.579919 3017 raft_consensus.cc:1261] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.580329 6065 consensus_queue.cc:1035] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.580448 540 raft_consensus.cc:1261] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:55.580725 6065 consensus_queue.cc:1035] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.580924 6208 consensus_queue.cc:1035] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:55.589109 6208 raft_consensus.cc:2932] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.590883 405 raft_consensus.cc:2932] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.590924 3017 raft_consensus.cc:2932] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.591089 540 consensus_queue.cc:237] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:55.592131 542 raft_consensus.cc:2932] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.594278 32732 catalog_manager.cc:5484] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.594564 803 raft_consensus.cc:1261] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.596141 2884 raft_consensus.cc:1261] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.596872 6051 consensus_queue.cc:1035] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:55.597455 476 consensus_peers.cc:485] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ad4e1b6dea1e4beeb4b82852ded83c5f. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.597916 6051 consensus_queue.cc:1035] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.600703 6051 raft_consensus.cc:2932] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:55.601646 673 consensus_queue.cc:237] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:55.603919 3017 raft_consensus.cc:1261] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.604389 405 raft_consensus.cc:1261] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.604476 803 raft_consensus.cc:2932] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:55.606045 5098 consensus_queue.cc:1035] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.606405 5098 consensus_queue.cc:1035] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.606307 32728 catalog_manager.cc:5484] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.609401 5711 raft_consensus.cc:2932] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.609725 405 raft_consensus.cc:2932] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.611320 6206 raft_consensus.cc:979] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: : Instructing follower 452ecbc085e54db49c3b01c20a96f860 to start an election
I20240502 14:00:55.611444 6206 raft_consensus.cc:1067] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Signalling peer 452ecbc085e54db49c3b01c20a96f860 to start an election
I20240502 14:00:55.612404 3019 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "909bc2eb63544db0b4b3a0764be7d4a1"
dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
from {username='slave'} at 127.26.25.133:36767
I20240502 14:00:55.612541 3019 raft_consensus.cc:482] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:55.612607 3019 raft_consensus.cc:3037] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.613608 3019 raft_consensus.cc:504] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:55.614070 3019 leader_election.cc:288] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:55.614559 3281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "909bc2eb63544db0b4b3a0764be7d4a1" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:55.614681 3281 raft_consensus.cc:3037] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.615001 2886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "909bc2eb63544db0b4b3a0764be7d4a1" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:55.615128 2886 raft_consensus.cc:3032] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:55.615168 2886 raft_consensus.cc:729] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:00:55.615345 2886 consensus_queue.cc:260] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:55.615567 2886 raft_consensus.cc:3037] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.615629 3281 raft_consensus.cc:2454] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 2.
I20240502 14:00:55.616596 2886 raft_consensus.cc:2454] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 2.
I20240502 14:00:55.616587 803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "909bc2eb63544db0b4b3a0764be7d4a1" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
I20240502 14:00:55.616695 803 raft_consensus.cc:3037] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.617610 803 raft_consensus.cc:2454] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 2.
I20240502 14:00:55.618346 2954 leader_election.cc:302] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:55.618759 6202 raft_consensus.cc:2781] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:55.618860 6202 raft_consensus.cc:686] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 2 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:55.619022 6202 consensus_queue.cc:237] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:55.620765 32734 catalog_manager.cc:5484] T a658bde2687344beabf4d8a0ff8988e2 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.622138 32728 catalog_manager.cc:5484] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 1 to 2, leader changed from 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 2 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.622239 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a658bde2687344beabf4d8a0ff8988e2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.625079 3017 raft_consensus.cc:2932] T a658bde2687344beabf4d8a0ff8988e2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:55.631893 520 tablet_service.cc:1514] Processing DeleteTablet for tablet a658bde2687344beabf4d8a0ff8988e2 with delete_type TABLET_DATA_TOMBSTONED (TS c351a89a5fe34e788b9d8064fbffa751 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:55.632146 6257 tablet_replica.cc:330] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:55.632300 6257 raft_consensus.cc:2229] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.632426 6257 raft_consensus.cc:2258] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.632988 6257 ts_tablet_manager.cc:1875] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.634588 405 consensus_queue.cc:237] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:55.635154 6257 ts_tablet_manager.cc:1888] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:55.635239 6257 log.cc:1196] T a658bde2687344beabf4d8a0ff8988e2 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/a658bde2687344beabf4d8a0ff8988e2
I20240502 14:00:55.636098 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet a658bde2687344beabf4d8a0ff8988e2 (table rebalancer_extra_table_66 [id=cb41deb677b547cd82087726509f0f21]) successfully deleted
I20240502 14:00:55.636929 2886 raft_consensus.cc:1261] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:55.637212 344 consensus_peers.cc:485] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: caffa941c89c48d1b3d36ca47dc49eb2. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.637408 6280 consensus_queue.cc:1035] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.638084 3017 raft_consensus.cc:1261] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.638554 6280 consensus_queue.cc:1035] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.640792 6280 raft_consensus.cc:2932] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.641258 2886 raft_consensus.cc:2932] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.642237 3019 raft_consensus.cc:2932] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.644541 32734 catalog_manager.cc:5484] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.647706 2884 raft_consensus.cc:2932] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:55.656862 672 consensus_queue.cc:237] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:55.659797 3150 raft_consensus.cc:1261] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:55.661571 5098 consensus_queue.cc:1035] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:55.663892 5098 raft_consensus.cc:2932] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.664070 406 raft_consensus.cc:1261] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:55.664165 3150 raft_consensus.cc:2932] T 3f9b1351891d4690b67f8eef000a902b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.665019 6221 consensus_queue.cc:1035] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:55.665705 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3f9b1351891d4690b67f8eef000a902b with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:55.667129 406 raft_consensus.cc:2932] T 3f9b1351891d4690b67f8eef000a902b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:55.667384 32728 catalog_manager.cc:5484] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.671869 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f9b1351891d4690b67f8eef000a902b with delete_type TABLET_DATA_TOMBSTONED (TS c351a89a5fe34e788b9d8064fbffa751 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:55.672187 6257 tablet_replica.cc:330] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:55.672329 6257 raft_consensus.cc:2229] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:55.672477 6257 raft_consensus.cc:2258] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:55.672632 406 consensus_queue.cc:237] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:55.673641 6257 ts_tablet_manager.cc:1875] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:55.674500 803 raft_consensus.cc:1261] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.674881 5715 consensus_queue.cc:1035] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:55.676288 341 consensus_peers.cc:485] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a176c4678de94ccc93b14d04fc46ea90. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.676652 3150 raft_consensus.cc:1261] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.676882 6172 raft_consensus.cc:2932] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:55.677145 6171 consensus_queue.cc:1035] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.677265 803 raft_consensus.cc:2932] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:55.678229 3150 raft_consensus.cc:2932] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:55.675778 6257 ts_tablet_manager.cc:1888] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:55.681322 6257 log.cc:1196] T 3f9b1351891d4690b67f8eef000a902b P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/3f9b1351891d4690b67f8eef000a902b
I20240502 14:00:55.681372 32734 catalog_manager.cc:5484] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:55.681917 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 3f9b1351891d4690b67f8eef000a902b (table rebalance_test_table_5 [id=4ce1c57cf9054191b787f9697c0c8f88]) successfully deleted
I20240502 14:00:55.694095 803 consensus_queue.cc:237] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:55.695919 672 raft_consensus.cc:1261] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.695981 406 raft_consensus.cc:1261] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.696287 6174 consensus_queue.cc:1035] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.696946 6174 consensus_queue.cc:1035] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:00:55.698485 739 consensus_peers.cc:485] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9486274cd39b4de18c37ab9f71f261c2. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.698909 6174 raft_consensus.cc:2932] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:55.702741 32731 catalog_manager.cc:5484] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.702834 406 raft_consensus.cc:2932] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:55.711138 672 raft_consensus.cc:2932] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:55.717756 3019 consensus_queue.cc:237] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:55.719177 2884 raft_consensus.cc:1261] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:55.719360 2953 consensus_peers.cc:485] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e93a3a3fca9042e9b8bdf3c60ec73a20. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.719341 3281 raft_consensus.cc:1261] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.720007 6068 consensus_queue.cc:1035] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.720355 6202 consensus_queue.cc:1035] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.722265 6235 raft_consensus.cc:2932] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:55.722779 2884 raft_consensus.cc:2932] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:55.723325 3284 raft_consensus.cc:2932] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:55.726487 32734 catalog_manager.cc:5484] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:55.730022 6282 raft_consensus.cc:979] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: : Instructing follower 478dfa53d7944f38b4e6cabee21b2d01 to start an election
I20240502 14:00:55.730145 6282 raft_consensus.cc:1067] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Signalling peer 478dfa53d7944f38b4e6cabee21b2d01 to start an election
I20240502 14:00:55.730998 803 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3"
dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
from {username='slave'} at 127.26.25.136:32835
I20240502 14:00:55.731184 803 raft_consensus.cc:482] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:55.731259 803 raft_consensus.cc:3037] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.732409 803 raft_consensus.cc:504] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:55.732986 803 leader_election.cc:288] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 2 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:55.733456 406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:00:55.733611 406 raft_consensus.cc:3037] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.734575 406 raft_consensus.cc:2454] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 2.
I20240502 14:00:55.735191 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:55.735189 3281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:55.735319 3281 raft_consensus.cc:3032] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:55.735319 3019 raft_consensus.cc:3037] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.735369 3281 raft_consensus.cc:729] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:55.735538 3281 consensus_queue.cc:260] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:55.735746 3281 raft_consensus.cc:3037] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.736421 3019 raft_consensus.cc:2454] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 2.
I20240502 14:00:55.736745 738 leader_election.cc:302] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:00:55.736974 6174 raft_consensus.cc:2781] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:55.737089 6174 raft_consensus.cc:686] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:55.737241 6174 consensus_queue.cc:237] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:55.739591 3281 raft_consensus.cc:2454] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 2.
I20240502 14:00:55.741425 32731 catalog_manager.cc:5484] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 1 to 2, leader changed from da3f8160557e454f8547f45e80c2f013 (127.26.25.136) to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.751271 2884 consensus_queue.cc:237] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:55.752610 3019 raft_consensus.cc:1261] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.752612 540 raft_consensus.cc:1261] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.753021 5358 consensus_queue.cc:1035] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.753211 5674 consensus_queue.cc:1035] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:55.753345 2822 consensus_peers.cc:485] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 10d6cc99575f4ebb8c980e9cb3e35145. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.755368 6206 raft_consensus.cc:2932] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.755920 540 raft_consensus.cc:2932] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.756129 3019 raft_consensus.cc:2932] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.759954 32731 catalog_manager.cc:5484] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.888722 3019 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "923a5d2ccd0448b381d88ced69ae8fe2"
dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38458
I20240502 14:00:55.888867 3019 raft_consensus.cc:595] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:55.918747 2884 consensus_queue.cc:237] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
W20240502 14:00:55.920270 2822 consensus_peers.cc:485] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 58a0cf9cd92a43ca8c481964707eda6d. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.920454 540 raft_consensus.cc:1261] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.920571 3019 raft_consensus.cc:1261] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.920989 5358 consensus_queue.cc:1035] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.921185 6206 consensus_queue.cc:1035] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.923074 5674 raft_consensus.cc:2932] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:55.923561 3019 raft_consensus.cc:2932] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:55.924943 540 raft_consensus.cc:2932] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:55.928606 32731 catalog_manager.cc:5484] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:55.934865 5637 raft_consensus.cc:979] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: : Instructing follower da3f8160557e454f8547f45e80c2f013 to start an election
I20240502 14:00:55.934970 5637 raft_consensus.cc:1067] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Signalling peer da3f8160557e454f8547f45e80c2f013 to start an election
I20240502 14:00:55.936602 540 consensus_queue.cc:237] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:55.936744 3281 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "923a5d2ccd0448b381d88ced69ae8fe2"
dest_uuid: "da3f8160557e454f8547f45e80c2f013"
from {username='slave'} at 127.26.25.134:42185
I20240502 14:00:55.936890 3281 raft_consensus.cc:482] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:55.936964 3281 raft_consensus.cc:3037] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.938087 3281 raft_consensus.cc:504] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:55.938257 805 raft_consensus.cc:1261] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:55.938508 477 consensus_peers.cc:485] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8568aa57fce64ccf89a36deb5b7b0f3b. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.938616 3281 leader_election.cc:288] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:55.938674 672 raft_consensus.cc:1261] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:55.938993 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923a5d2ccd0448b381d88ced69ae8fe2" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:00:55.939114 2884 raft_consensus.cc:3037] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.939299 3149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923a5d2ccd0448b381d88ced69ae8fe2" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "2b6796cf0e2e496e9dd641beb432f485"
I20240502 14:00:55.939407 3149 raft_consensus.cc:3037] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.939500 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923a5d2ccd0448b381d88ced69ae8fe2" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:00:55.939610 3019 raft_consensus.cc:3032] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:55.939638 3019 raft_consensus.cc:729] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:00:55.939785 3019 consensus_queue.cc:260] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:55.939997 3019 raft_consensus.cc:3037] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:55.940192 2884 raft_consensus.cc:2454] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 2.
I20240502 14:00:55.940524 3149 raft_consensus.cc:2454] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 2.
I20240502 14:00:55.940707 2884 raft_consensus.cc:1226] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Rejecting Update request from peer 452ecbc085e54db49c3b01c20a96f860 for earlier term 1. Current term is 2. Ops: []
I20240502 14:00:55.940752 5655 consensus_queue.cc:1035] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.940949 3019 raft_consensus.cc:2454] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 2.
I20240502 14:00:55.941085 3219 leader_election.cc:302] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2b6796cf0e2e496e9dd641beb432f485, 4bcd7712aa844d98977a7d7e605c31f5, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:55.941102 5655 consensus_queue.cc:1035] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:55.941257 6282 raft_consensus.cc:2781] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:55.941362 6282 raft_consensus.cc:686] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [term 2 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:00:55.941509 6282 consensus_queue.cc:237] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:55.943207 6600 ts_tablet_manager.cc:902] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:55.943102 6200 raft_consensus.cc:2932] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.943764 6600 tablet_copy_client.cc:308] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:55.943921 805 raft_consensus.cc:2932] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.944200 694 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 7113a6da4229422099dbd604a560fb95 from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:37651)
I20240502 14:00:55.944303 694 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 7113a6da4229422099dbd604a560fb95 from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:37651: session id = 2b6796cf0e2e496e9dd641beb432f485-7113a6da4229422099dbd604a560fb95
I20240502 14:00:55.944429 32734 catalog_manager.cc:5484] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 1 to 2, leader changed from 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 2 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:55.945258 694 tablet_copy_source_session.cc:205] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:55.947357 32728 catalog_manager.cc:5484] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.947598 672 raft_consensus.cc:2932] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:55.954285 6600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7113a6da4229422099dbd604a560fb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:55.957513 6600 tablet_copy_client.cc:783] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:55.957770 6600 tablet_copy_client.cc:647] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:55.960146 6600 tablet_copy_client.cc:523] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:55.961313 6600 tablet_bootstrap.cc:492] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:55.961807 2886 consensus_queue.cc:237] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:55.963260 3019 raft_consensus.cc:1261] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:55.963445 2819 consensus_peers.cc:485] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 184c03808a1e470dba9c6bfa8e04774a. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.963541 3281 raft_consensus.cc:1261] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.963954 5358 consensus_queue.cc:1035] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.964161 5674 consensus_queue.cc:1035] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.966303 6206 raft_consensus.cc:2932] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:55.966794 3281 raft_consensus.cc:2932] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:55.968859 3019 raft_consensus.cc:2932] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:55.970206 32731 catalog_manager.cc:5484] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.980800 3149 consensus_queue.cc:237] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
W20240502 14:00:55.982744 3087 consensus_peers.cc:485] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): Couldn't send request to peer 4bcd7712aa844d98977a7d7e605c31f5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 564ba4de769e45af8de10fe8c86eef31. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:55.982853 406 raft_consensus.cc:1261] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.983353 6270 consensus_queue.cc:1035] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.983650 3281 raft_consensus.cc:1261] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:55.984449 6270 consensus_queue.cc:1035] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:55.985654 6270 raft_consensus.cc:2932] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:55.987877 6600 tablet_bootstrap.cc:492] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:55.988278 6600 tablet_bootstrap.cc:492] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:55.988546 6600 ts_tablet_manager.cc:1367] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.027s user 0.027s sys 0.000s
I20240502 14:00:55.988674 406 raft_consensus.cc:2932] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:55.989163 6600 raft_consensus.cc:348] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:55.989329 6600 raft_consensus.cc:729] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:55.989478 6600 consensus_queue.cc:260] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:55.989950 6600 ts_tablet_manager.cc:1398] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:55.990422 694 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-7113a6da4229422099dbd604a560fb95 received from {username='slave'} at 127.26.25.135:37651
I20240502 14:00:55.990504 694 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-7113a6da4229422099dbd604a560fb95 on tablet 7113a6da4229422099dbd604a560fb95 with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:55.990366 32734 catalog_manager.cc:5484] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:55.994246 3281 raft_consensus.cc:2932] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:56.003113 672 consensus_queue.cc:237] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:56.004583 805 raft_consensus.cc:1261] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:56.004729 609 consensus_peers.cc:485] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 59dd5ee0b3554ac493ce2d4d15a99abd. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:56.005039 6221 consensus_queue.cc:1035] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:56.005180 405 raft_consensus.cc:1261] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:56.005542 6221 consensus_queue.cc:1035] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:56.005661 6633 ts_tablet_manager.cc:902] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:56.006183 6633 tablet_copy_client.cc:308] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35251
I20240502 14:00:56.006593 562 tablet_copy_service.cc:140] P c351a89a5fe34e788b9d8064fbffa751: Received BeginTabletCopySession request for tablet ad4e1b6dea1e4beeb4b82852ded83c5f from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:42705)
I20240502 14:00:56.006721 562 tablet_copy_service.cc:161] P c351a89a5fe34e788b9d8064fbffa751: Beginning new tablet copy session on tablet ad4e1b6dea1e4beeb4b82852ded83c5f from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:42705: session id = f003b732dd0f46958262b42f8d20305e-ad4e1b6dea1e4beeb4b82852ded83c5f
I20240502 14:00:56.007293 5711 raft_consensus.cc:2932] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:56.007781 562 tablet_copy_source_session.cc:205] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.007746 405 raft_consensus.cc:2932] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:56.011453 32731 catalog_manager.cc:5484] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:56.012488 805 raft_consensus.cc:2932] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:56.014058 6641 ts_tablet_manager.cc:902] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:56.014590 6641 tablet_copy_client.cc:308] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:56.015168 694 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet e5bed83e31be4129a19eb8c347514b80 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:47487)
I20240502 14:00:56.015256 694 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet e5bed83e31be4129a19eb8c347514b80 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:47487: session id = da3f8160557e454f8547f45e80c2f013-e5bed83e31be4129a19eb8c347514b80
I20240502 14:00:56.017114 694 tablet_copy_source_session.cc:205] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.017614 6640 ts_tablet_manager.cc:902] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751: Initiating tablet copy from peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:56.018169 6640 tablet_copy_client.cc:308] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.134:39061
I20240502 14:00:56.018790 3039 tablet_copy_service.cc:140] P 452ecbc085e54db49c3b01c20a96f860: Received BeginTabletCopySession request for tablet 792bc86bad314a2bbf4bba3001c78894 from peer c351a89a5fe34e788b9d8064fbffa751 ({username='slave'} at 127.26.25.130:33995)
I20240502 14:00:56.018898 3039 tablet_copy_service.cc:161] P 452ecbc085e54db49c3b01c20a96f860: Beginning new tablet copy session on tablet 792bc86bad314a2bbf4bba3001c78894 from peer c351a89a5fe34e788b9d8064fbffa751 at {username='slave'} at 127.26.25.130:33995: session id = c351a89a5fe34e788b9d8064fbffa751-792bc86bad314a2bbf4bba3001c78894
I20240502 14:00:56.019848 3039 tablet_copy_source_session.cc:205] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.021693 6640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 792bc86bad314a2bbf4bba3001c78894. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.023936 6640 tablet_copy_client.cc:783] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.024155 6640 tablet_copy_client.cc:647] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.024178 6633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad4e1b6dea1e4beeb4b82852ded83c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.027763 6641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5bed83e31be4129a19eb8c347514b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.028753 6640 tablet_copy_client.cc:523] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.030050 6640 tablet_bootstrap.cc:492] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:56.034947 6633 tablet_copy_client.cc:783] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.035182 6633 tablet_copy_client.cc:647] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.035313 6641 tablet_copy_client.cc:783] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.035496 6641 tablet_copy_client.cc:647] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.036480 6633 tablet_copy_client.cc:523] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.039749 6641 tablet_copy_client.cc:523] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.040328 6633 tablet_bootstrap.cc:492] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:56.041137 6641 tablet_bootstrap.cc:492] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:56.043612 6640 tablet_bootstrap.cc:492] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.043843 6640 tablet_bootstrap.cc:492] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap complete.
I20240502 14:00:56.044019 6640 ts_tablet_manager.cc:1367] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20240502 14:00:56.044446 6640 raft_consensus.cc:348] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:56.044566 6640 raft_consensus.cc:729] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: LEARNER
I20240502 14:00:56.044657 6640 consensus_queue.cc:260] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:56.045017 6640 ts_tablet_manager.cc:1398] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:56.045547 3039 tablet_copy_service.cc:342] P 452ecbc085e54db49c3b01c20a96f860: Request end of tablet copy session c351a89a5fe34e788b9d8064fbffa751-792bc86bad314a2bbf4bba3001c78894 received from {username='slave'} at 127.26.25.130:33995
I20240502 14:00:56.045620 3039 tablet_copy_service.cc:433] P 452ecbc085e54db49c3b01c20a96f860: ending tablet copy session c351a89a5fe34e788b9d8064fbffa751-792bc86bad314a2bbf4bba3001c78894 on tablet 792bc86bad314a2bbf4bba3001c78894 with peer c351a89a5fe34e788b9d8064fbffa751
I20240502 14:00:56.055760 6633 tablet_bootstrap.cc:492] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.056092 6633 tablet_bootstrap.cc:492] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:00:56.056288 6633 ts_tablet_manager.cc:1367] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.004s
I20240502 14:00:56.056761 6633 raft_consensus.cc:348] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:56.056888 6633 raft_consensus.cc:729] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:00:56.056979 6633 consensus_queue.cc:260] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:56.057279 6633 ts_tablet_manager.cc:1398] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:56.057745 562 tablet_copy_service.cc:342] P c351a89a5fe34e788b9d8064fbffa751: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-ad4e1b6dea1e4beeb4b82852ded83c5f received from {username='slave'} at 127.26.25.129:42705
I20240502 14:00:56.057819 562 tablet_copy_service.cc:433] P c351a89a5fe34e788b9d8064fbffa751: ending tablet copy session f003b732dd0f46958262b42f8d20305e-ad4e1b6dea1e4beeb4b82852ded83c5f on tablet ad4e1b6dea1e4beeb4b82852ded83c5f with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:00:56.092720 6641 tablet_bootstrap.cc:492] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.093077 6641 tablet_bootstrap.cc:492] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:00:56.093276 6641 ts_tablet_manager.cc:1367] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.052s user 0.029s sys 0.000s
I20240502 14:00:56.093669 6675 ts_tablet_manager.cc:902] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860: Initiating tablet copy from peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:56.093739 6641 raft_consensus.cc:348] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:56.093884 6641 raft_consensus.cc:729] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:00:56.094012 6641 consensus_queue.cc:260] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:56.094311 6641 ts_tablet_manager.cc:1398] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:56.094592 6674 ts_tablet_manager.cc:902] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:56.095204 6674 tablet_copy_client.cc:308] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:56.096843 694 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-e5bed83e31be4129a19eb8c347514b80 received from {username='slave'} at 127.26.25.136:47487
I20240502 14:00:56.096928 694 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session da3f8160557e454f8547f45e80c2f013-e5bed83e31be4129a19eb8c347514b80 on tablet e5bed83e31be4129a19eb8c347514b80 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:00:56.097236 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet be795fbe277b4e9fac8561d6589b7c05 from peer 4bcd7712aa844d98977a7d7e605c31f5 ({username='slave'} at 127.26.25.133:46031)
I20240502 14:00:56.097332 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet be795fbe277b4e9fac8561d6589b7c05 from peer 4bcd7712aa844d98977a7d7e605c31f5 at {username='slave'} at 127.26.25.133:46031: session id = 4bcd7712aa844d98977a7d7e605c31f5-be795fbe277b4e9fac8561d6589b7c05
I20240502 14:00:56.097770 6675 tablet_copy_client.cc:308] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:44715
I20240502 14:00:56.098313 693 tablet_copy_source_session.cc:205] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.099135 430 tablet_copy_service.cc:140] P f003b732dd0f46958262b42f8d20305e: Received BeginTabletCopySession request for tablet 1bdc7b131596496aa6779885fe5d9433 from peer 452ecbc085e54db49c3b01c20a96f860 ({username='slave'} at 127.26.25.134:36165)
I20240502 14:00:56.099248 430 tablet_copy_service.cc:161] P f003b732dd0f46958262b42f8d20305e: Beginning new tablet copy session on tablet 1bdc7b131596496aa6779885fe5d9433 from peer 452ecbc085e54db49c3b01c20a96f860 at {username='slave'} at 127.26.25.134:36165: session id = 452ecbc085e54db49c3b01c20a96f860-1bdc7b131596496aa6779885fe5d9433
I20240502 14:00:56.099393 6674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be795fbe277b4e9fac8561d6589b7c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.100272 430 tablet_copy_source_session.cc:205] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.101626 6674 tablet_copy_client.cc:783] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.101791 6674 tablet_copy_client.cc:647] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.103096 6674 tablet_copy_client.cc:523] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.103133 6675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdc7b131596496aa6779885fe5d9433. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.105875 6675 tablet_copy_client.cc:783] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.106040 6675 tablet_copy_client.cc:647] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.106721 6674 tablet_bootstrap.cc:492] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:56.109059 3281 raft_consensus.cc:1261] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:00:56.109553 6675 tablet_copy_client.cc:523] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.109529 6068 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.111013 6675 tablet_bootstrap.cc:492] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:56.125217 3019 raft_consensus.cc:1261] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:00:56.125953 5429 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.136333 6640 ts_tablet_manager.cc:902] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751: Initiating tablet copy from peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:56.136829 6640 tablet_copy_client.cc:308] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:44715
I20240502 14:00:56.137310 430 tablet_copy_service.cc:140] P f003b732dd0f46958262b42f8d20305e: Received BeginTabletCopySession request for tablet a176c4678de94ccc93b14d04fc46ea90 from peer c351a89a5fe34e788b9d8064fbffa751 ({username='slave'} at 127.26.25.130:42179)
I20240502 14:00:56.137434 430 tablet_copy_service.cc:161] P f003b732dd0f46958262b42f8d20305e: Beginning new tablet copy session on tablet a176c4678de94ccc93b14d04fc46ea90 from peer c351a89a5fe34e788b9d8064fbffa751 at {username='slave'} at 127.26.25.130:42179: session id = c351a89a5fe34e788b9d8064fbffa751-a176c4678de94ccc93b14d04fc46ea90
I20240502 14:00:56.138533 430 tablet_copy_source_session.cc:205] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.140350 6674 tablet_bootstrap.cc:492] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.140681 6674 tablet_bootstrap.cc:492] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap complete.
I20240502 14:00:56.140857 6674 ts_tablet_manager.cc:1367] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.036s user 0.023s sys 0.009s
I20240502 14:00:56.140972 6640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a176c4678de94ccc93b14d04fc46ea90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.141300 6674 raft_consensus.cc:348] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:00:56.141427 6674 raft_consensus.cc:729] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: LEARNER
I20240502 14:00:56.141520 6674 consensus_queue.cc:260] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:00:56.141796 6674 ts_tablet_manager.cc:1398] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:56.143110 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-be795fbe277b4e9fac8561d6589b7c05 received from {username='slave'} at 127.26.25.133:46031
I20240502 14:00:56.143191 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-be795fbe277b4e9fac8561d6589b7c05 on tablet be795fbe277b4e9fac8561d6589b7c05 with peer 4bcd7712aa844d98977a7d7e605c31f5
I20240502 14:00:56.143198 6640 tablet_copy_client.cc:783] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.143354 6640 tablet_copy_client.cc:647] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.144428 6675 tablet_bootstrap.cc:492] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.144752 6675 tablet_bootstrap.cc:492] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap complete.
I20240502 14:00:56.144929 6675 ts_tablet_manager.cc:1367] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.034s user 0.027s sys 0.000s
I20240502 14:00:56.145483 6600 ts_tablet_manager.cc:902] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:56.145931 6600 tablet_copy_client.cc:308] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:44715
I20240502 14:00:56.146176 6633 ts_tablet_manager.cc:902] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:56.146203 6640 tablet_copy_client.cc:523] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.146626 6633 tablet_copy_client.cc:308] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.134:39061
I20240502 14:00:56.147176 6675 raft_consensus.cc:348] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:56.147330 6675 raft_consensus.cc:729] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: LEARNER
I20240502 14:00:56.147184 3039 tablet_copy_service.cc:140] P 452ecbc085e54db49c3b01c20a96f860: Received BeginTabletCopySession request for tablet e93a3a3fca9042e9b8bdf3c60ec73a20 from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:35347)
I20240502 14:00:56.147478 3039 tablet_copy_service.cc:161] P 452ecbc085e54db49c3b01c20a96f860: Beginning new tablet copy session on tablet e93a3a3fca9042e9b8bdf3c60ec73a20 from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:35347: session id = f003b732dd0f46958262b42f8d20305e-e93a3a3fca9042e9b8bdf3c60ec73a20
I20240502 14:00:56.147451 6675 consensus_queue.cc:260] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:56.147549 6640 tablet_bootstrap.cc:492] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:56.147859 6675 ts_tablet_manager.cc:1398] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:56.148597 3039 tablet_copy_source_session.cc:205] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.149567 6633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93a3a3fca9042e9b8bdf3c60ec73a20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.150522 430 tablet_copy_service.cc:342] P f003b732dd0f46958262b42f8d20305e: Request end of tablet copy session 452ecbc085e54db49c3b01c20a96f860-1bdc7b131596496aa6779885fe5d9433 received from {username='slave'} at 127.26.25.134:36165
I20240502 14:00:56.150584 430 tablet_copy_service.cc:433] P f003b732dd0f46958262b42f8d20305e: ending tablet copy session 452ecbc085e54db49c3b01c20a96f860-1bdc7b131596496aa6779885fe5d9433 on tablet 1bdc7b131596496aa6779885fe5d9433 with peer 452ecbc085e54db49c3b01c20a96f860
I20240502 14:00:56.151129 429 tablet_copy_service.cc:140] P f003b732dd0f46958262b42f8d20305e: Received BeginTabletCopySession request for tablet caffa941c89c48d1b3d36ca47dc49eb2 from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:55895)
I20240502 14:00:56.151209 429 tablet_copy_service.cc:161] P f003b732dd0f46958262b42f8d20305e: Beginning new tablet copy session on tablet caffa941c89c48d1b3d36ca47dc49eb2 from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:55895: session id = 2b6796cf0e2e496e9dd641beb432f485-caffa941c89c48d1b3d36ca47dc49eb2
I20240502 14:00:56.151844 6633 tablet_copy_client.cc:783] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.152000 6633 tablet_copy_client.cc:647] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.152103 6640 tablet_bootstrap.cc:492] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.152165 429 tablet_copy_source_session.cc:205] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.152357 6640 tablet_bootstrap.cc:492] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap complete.
I20240502 14:00:56.152524 6640 ts_tablet_manager.cc:1367] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 14:00:56.153275 6640 raft_consensus.cc:348] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:56.153435 6640 raft_consensus.cc:729] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: LEARNER
I20240502 14:00:56.153546 6640 consensus_queue.cc:260] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:56.153921 6640 ts_tablet_manager.cc:1398] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:56.154247 6600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caffa941c89c48d1b3d36ca47dc49eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.159049 429 tablet_copy_service.cc:342] P f003b732dd0f46958262b42f8d20305e: Request end of tablet copy session c351a89a5fe34e788b9d8064fbffa751-a176c4678de94ccc93b14d04fc46ea90 received from {username='slave'} at 127.26.25.130:42179
I20240502 14:00:56.159127 429 tablet_copy_service.cc:433] P f003b732dd0f46958262b42f8d20305e: ending tablet copy session c351a89a5fe34e788b9d8064fbffa751-a176c4678de94ccc93b14d04fc46ea90 on tablet a176c4678de94ccc93b14d04fc46ea90 with peer c351a89a5fe34e788b9d8064fbffa751
I20240502 14:00:56.161106 6633 tablet_copy_client.cc:523] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.161777 6600 tablet_copy_client.cc:783] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.161936 6600 tablet_copy_client.cc:647] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.162889 6633 tablet_bootstrap.cc:492] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:56.163041 6600 tablet_copy_client.cc:523] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.164422 6600 tablet_bootstrap.cc:492] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:56.166896 6633 tablet_bootstrap.cc:492] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.167167 6633 tablet_bootstrap.cc:492] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:00:56.167332 6633 ts_tablet_manager.cc:1367] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240502 14:00:56.167814 6633 raft_consensus.cc:348] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:56.167994 6633 raft_consensus.cc:729] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:00:56.168103 6633 consensus_queue.cc:260] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:56.168376 6633 ts_tablet_manager.cc:1398] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:56.168777 3039 tablet_copy_service.cc:342] P 452ecbc085e54db49c3b01c20a96f860: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-e93a3a3fca9042e9b8bdf3c60ec73a20 received from {username='slave'} at 127.26.25.129:35347
I20240502 14:00:56.168969 3039 tablet_copy_service.cc:433] P 452ecbc085e54db49c3b01c20a96f860: ending tablet copy session f003b732dd0f46958262b42f8d20305e-e93a3a3fca9042e9b8bdf3c60ec73a20 on tablet e93a3a3fca9042e9b8bdf3c60ec73a20 with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:00:56.170490 6600 tablet_bootstrap.cc:492] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.170751 6600 tablet_bootstrap.cc:492] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:56.170890 6600 ts_tablet_manager.cc:1367] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20240502 14:00:56.171351 6600 raft_consensus.cc:348] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:56.171474 6600 raft_consensus.cc:729] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:56.171569 6600 consensus_queue.cc:260] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:56.171895 6600 ts_tablet_manager.cc:1398] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:56.172322 429 tablet_copy_service.cc:342] P f003b732dd0f46958262b42f8d20305e: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-caffa941c89c48d1b3d36ca47dc49eb2 received from {username='slave'} at 127.26.25.135:55895
I20240502 14:00:56.172396 429 tablet_copy_service.cc:433] P f003b732dd0f46958262b42f8d20305e: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-caffa941c89c48d1b3d36ca47dc49eb2 on tablet caffa941c89c48d1b3d36ca47dc49eb2 with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:56.197211 2884 raft_consensus.cc:1261] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:00:56.197675 6068 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.200263 805 raft_consensus.cc:1261] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:00:56.200659 5637 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.214691 6641 ts_tablet_manager.cc:902] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:56.215219 6641 tablet_copy_client.cc:308] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.132:33443
I20240502 14:00:56.215703 826 tablet_copy_service.cc:140] P 478dfa53d7944f38b4e6cabee21b2d01: Received BeginTabletCopySession request for tablet 9486274cd39b4de18c37ab9f71f261c2 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:32835)
I20240502 14:00:56.215847 826 tablet_copy_service.cc:161] P 478dfa53d7944f38b4e6cabee21b2d01: Beginning new tablet copy session on tablet 9486274cd39b4de18c37ab9f71f261c2 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:32835: session id = da3f8160557e454f8547f45e80c2f013-9486274cd39b4de18c37ab9f71f261c2
I20240502 14:00:56.216998 826 tablet_copy_source_session.cc:205] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.218261 6641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9486274cd39b4de18c37ab9f71f261c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.220274 6641 tablet_copy_client.cc:783] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.220458 6641 tablet_copy_client.cc:647] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.222681 6641 tablet_copy_client.cc:523] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.223994 6641 tablet_bootstrap.cc:492] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:56.248801 6641 tablet_bootstrap.cc:492] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.249248 6641 tablet_bootstrap.cc:492] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:00:56.249536 6641 ts_tablet_manager.cc:1367] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.026s user 0.023s sys 0.001s
I20240502 14:00:56.250067 6641 raft_consensus.cc:348] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:56.250281 6641 raft_consensus.cc:729] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:00:56.250479 6641 consensus_queue.cc:260] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:56.252082 6641 ts_tablet_manager.cc:1398] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:56.252575 826 tablet_copy_service.cc:342] P 478dfa53d7944f38b4e6cabee21b2d01: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-9486274cd39b4de18c37ab9f71f261c2 received from {username='slave'} at 127.26.25.136:32835
I20240502 14:00:56.252665 826 tablet_copy_service.cc:433] P 478dfa53d7944f38b4e6cabee21b2d01: ending tablet copy session da3f8160557e454f8547f45e80c2f013-9486274cd39b4de18c37ab9f71f261c2 on tablet 9486274cd39b4de18c37ab9f71f261c2 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:00:56.253216 6600 ts_tablet_manager.cc:902] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:56.253751 6600 tablet_copy_client.cc:308] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:00:56.254226 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 10d6cc99575f4ebb8c980e9cb3e35145 from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:33043)
I20240502 14:00:56.254324 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 10d6cc99575f4ebb8c980e9cb3e35145 from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:33043: session id = 2b6796cf0e2e496e9dd641beb432f485-10d6cc99575f4ebb8c980e9cb3e35145
I20240502 14:00:56.255268 2906 tablet_copy_source_session.cc:205] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.256239 6600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10d6cc99575f4ebb8c980e9cb3e35145. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.258460 6600 tablet_copy_client.cc:783] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.258632 6600 tablet_copy_client.cc:647] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.260006 6600 tablet_copy_client.cc:523] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.261337 6600 tablet_bootstrap.cc:492] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:56.264844 6600 tablet_bootstrap.cc:492] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.265077 6600 tablet_bootstrap.cc:492] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:56.265242 6600 ts_tablet_manager.cc:1367] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 14:00:56.265666 6600 raft_consensus.cc:348] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:56.265799 6600 raft_consensus.cc:729] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:56.265892 6600 consensus_queue.cc:260] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:56.266232 6600 ts_tablet_manager.cc:1398] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:56.267103 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-10d6cc99575f4ebb8c980e9cb3e35145 received from {username='slave'} at 127.26.25.135:33043
I20240502 14:00:56.267184 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-10d6cc99575f4ebb8c980e9cb3e35145 on tablet 10d6cc99575f4ebb8c980e9cb3e35145 with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:56.319713 6793 ts_tablet_manager.cc:902] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:56.320341 6793 tablet_copy_client.cc:308] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:00:56.320951 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 58a0cf9cd92a43ca8c481964707eda6d from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:41861)
I20240502 14:00:56.321067 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 58a0cf9cd92a43ca8c481964707eda6d from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:41861: session id = 478dfa53d7944f38b4e6cabee21b2d01-58a0cf9cd92a43ca8c481964707eda6d
I20240502 14:00:56.322124 2906 tablet_copy_source_session.cc:205] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.323040 6793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58a0cf9cd92a43ca8c481964707eda6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.323454 3284 raft_consensus.cc:1261] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:00:56.323918 5429 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.325736 6793 tablet_copy_client.cc:783] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.325902 6793 tablet_copy_client.cc:647] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.327318 405 raft_consensus.cc:1261] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:00:56.327898 6174 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.329187 6793 tablet_copy_client.cc:523] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.330910 6793 tablet_bootstrap.cc:492] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:56.335350 6793 tablet_bootstrap.cc:492] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.335733 6793 tablet_bootstrap.cc:492] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:00:56.336059 6793 ts_tablet_manager.cc:1367] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240502 14:00:56.336581 6793 raft_consensus.cc:348] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:56.336715 6793 raft_consensus.cc:729] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:00:56.336822 6793 consensus_queue.cc:260] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:56.337086 6793 ts_tablet_manager.cc:1398] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:56.337555 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-58a0cf9cd92a43ca8c481964707eda6d received from {username='slave'} at 127.26.25.132:41861
I20240502 14:00:56.337646 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-58a0cf9cd92a43ca8c481964707eda6d on tablet 58a0cf9cd92a43ca8c481964707eda6d with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:00:56.362346 3149 raft_consensus.cc:1261] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 2 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:00:56.362860 6282 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.390306 6674 ts_tablet_manager.cc:902] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:56.390828 6674 tablet_copy_client.cc:308] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:00:56.391408 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet 564ba4de769e45af8de10fe8c86eef31 from peer 4bcd7712aa844d98977a7d7e605c31f5 ({username='slave'} at 127.26.25.133:59463)
I20240502 14:00:56.391520 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet 564ba4de769e45af8de10fe8c86eef31 from peer 4bcd7712aa844d98977a7d7e605c31f5 at {username='slave'} at 127.26.25.133:59463: session id = 4bcd7712aa844d98977a7d7e605c31f5-564ba4de769e45af8de10fe8c86eef31
I20240502 14:00:56.392474 3172 tablet_copy_source_session.cc:205] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.393370 6674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 564ba4de769e45af8de10fe8c86eef31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.395196 6674 tablet_copy_client.cc:783] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.395366 6674 tablet_copy_client.cc:647] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.396896 6674 tablet_copy_client.cc:523] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.397998 6674 tablet_bootstrap.cc:492] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:56.403373 6674 tablet_bootstrap.cc:492] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.403592 6674 tablet_bootstrap.cc:492] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap complete.
I20240502 14:00:56.403759 6674 ts_tablet_manager.cc:1367] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20240502 14:00:56.404170 6674 raft_consensus.cc:348] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:00:56.404278 6674 raft_consensus.cc:729] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: LEARNER
I20240502 14:00:56.404366 6674 consensus_queue.cc:260] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:00:56.404611 6674 ts_tablet_manager.cc:1398] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:56.405607 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-564ba4de769e45af8de10fe8c86eef31 received from {username='slave'} at 127.26.25.133:59463
I20240502 14:00:56.405687 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-564ba4de769e45af8de10fe8c86eef31 on tablet 564ba4de769e45af8de10fe8c86eef31 with peer 4bcd7712aa844d98977a7d7e605c31f5
I20240502 14:00:56.418998 6600 ts_tablet_manager.cc:902] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:56.419487 6600 tablet_copy_client.cc:308] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:56.419948 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 59dd5ee0b3554ac493ce2d4d15a99abd from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:37651)
I20240502 14:00:56.420056 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 59dd5ee0b3554ac493ce2d4d15a99abd from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:37651: session id = 2b6796cf0e2e496e9dd641beb432f485-59dd5ee0b3554ac493ce2d4d15a99abd
I20240502 14:00:56.421111 693 tablet_copy_source_session.cc:205] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.421875 6600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59dd5ee0b3554ac493ce2d4d15a99abd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.423732 6600 tablet_copy_client.cc:783] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.423919 6600 tablet_copy_client.cc:647] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.425194 6600 tablet_copy_client.cc:523] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.426441 6600 tablet_bootstrap.cc:492] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:56.427184 541 raft_consensus.cc:1203] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:56.430013 6600 tablet_bootstrap.cc:492] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.430240 6600 tablet_bootstrap.cc:492] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:56.430392 6600 ts_tablet_manager.cc:1367] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20240502 14:00:56.430819 6600 raft_consensus.cc:348] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:56.430960 6600 raft_consensus.cc:729] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:56.431069 6600 consensus_queue.cc:260] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:56.431376 6600 ts_tablet_manager.cc:1398] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:56.431921 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-59dd5ee0b3554ac493ce2d4d15a99abd received from {username='slave'} at 127.26.25.135:37651
I20240502 14:00:56.432013 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-59dd5ee0b3554ac493ce2d4d15a99abd on tablet 59dd5ee0b3554ac493ce2d4d15a99abd with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:56.440304 2884 raft_consensus.cc:1261] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:00:56.440778 6282 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.443109 3019 raft_consensus.cc:1261] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:00:56.443509 5104 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.448698 3284 consensus_queue.cc:237] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:56.450264 2884 raft_consensus.cc:1261] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:00:56.450423 3149 raft_consensus.cc:1261] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 2 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:00:56.450920 5104 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:56.451192 5104 consensus_queue.cc:1035] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:56.452597 6282 raft_consensus.cc:2932] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:56.453105 2886 raft_consensus.cc:2932] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:56.453279 3149 raft_consensus.cc:2932] T 923a5d2ccd0448b381d88ced69ae8fe2 P 2b6796cf0e2e496e9dd641beb432f485 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:56.454375 32719 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 923a5d2ccd0448b381d88ced69ae8fe2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:56.456729 32728 catalog_manager.cc:5484] T 923a5d2ccd0448b381d88ced69ae8fe2 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 5, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 2 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
W20240502 14:00:56.457185 32719 catalog_manager.cc:5007] ChangeConfig:REMOVE_PEER RPC for tablet 923a5d2ccd0448b381d88ced69ae8fe2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 5
I20240502 14:00:56.461274 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 923a5d2ccd0448b381d88ced69ae8fe2 with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:56.461688 6817 tablet_replica.cc:330] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:56.461841 6817 raft_consensus.cc:2229] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:56.461947 6817 raft_consensus.cc:2258] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:56.462496 6817 ts_tablet_manager.cc:1875] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:56.464537 6817 ts_tablet_manager.cc:1888] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20240502 14:00:56.464617 6817 log.cc:1196] T 923a5d2ccd0448b381d88ced69ae8fe2 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/923a5d2ccd0448b381d88ced69ae8fe2
I20240502 14:00:56.465126 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 923a5d2ccd0448b381d88ced69ae8fe2 (table rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]) successfully deleted
I20240502 14:00:56.482074 3284 raft_consensus.cc:1203] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:56.510577 405 raft_consensus.cc:1203] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:56.527869 3149 raft_consensus.cc:1203] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:56.540524 3019 raft_consensus.cc:1203] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:56.549530 6600 ts_tablet_manager.cc:902] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:56.550011 6600 tablet_copy_client.cc:308] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35251
I20240502 14:00:56.550565 562 tablet_copy_service.cc:140] P c351a89a5fe34e788b9d8064fbffa751: Received BeginTabletCopySession request for tablet 8568aa57fce64ccf89a36deb5b7b0f3b from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:53809)
I20240502 14:00:56.550680 562 tablet_copy_service.cc:161] P c351a89a5fe34e788b9d8064fbffa751: Beginning new tablet copy session on tablet 8568aa57fce64ccf89a36deb5b7b0f3b from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:53809: session id = 2b6796cf0e2e496e9dd641beb432f485-8568aa57fce64ccf89a36deb5b7b0f3b
I20240502 14:00:56.552138 562 tablet_copy_source_session.cc:205] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.553103 6600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8568aa57fce64ccf89a36deb5b7b0f3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.555604 6600 tablet_copy_client.cc:783] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.555768 6600 tablet_copy_client.cc:647] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.557693 6640 ts_tablet_manager.cc:902] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:56.558138 6640 tablet_copy_client.cc:308] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:00:56.558823 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 184c03808a1e470dba9c6bfa8e04774a from peer c351a89a5fe34e788b9d8064fbffa751 ({username='slave'} at 127.26.25.130:46819)
I20240502 14:00:56.558935 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 184c03808a1e470dba9c6bfa8e04774a from peer c351a89a5fe34e788b9d8064fbffa751 at {username='slave'} at 127.26.25.130:46819: session id = c351a89a5fe34e788b9d8064fbffa751-184c03808a1e470dba9c6bfa8e04774a
I20240502 14:00:56.559113 6600 tablet_copy_client.cc:523] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.560101 2906 tablet_copy_source_session.cc:205] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:56.560562 6600 tablet_bootstrap.cc:492] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:56.560986 6640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 184c03808a1e470dba9c6bfa8e04774a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:56.563218 6640 tablet_copy_client.cc:783] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:56.563366 6640 tablet_copy_client.cc:647] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:56.563987 3149 raft_consensus.cc:1203] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:56.565616 6640 tablet_copy_client.cc:523] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:56.566862 6640 tablet_bootstrap.cc:492] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:56.572219 6640 tablet_bootstrap.cc:492] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.572512 6640 tablet_bootstrap.cc:492] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751: Bootstrap complete.
I20240502 14:00:56.572682 6640 ts_tablet_manager.cc:1367] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20240502 14:00:56.573181 6640 raft_consensus.cc:348] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:56.573401 6640 raft_consensus.cc:729] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: LEARNER
I20240502 14:00:56.573527 6640 consensus_queue.cc:260] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:56.574884 6640 ts_tablet_manager.cc:1398] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:56.575356 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session c351a89a5fe34e788b9d8064fbffa751-184c03808a1e470dba9c6bfa8e04774a received from {username='slave'} at 127.26.25.130:46819
I20240502 14:00:56.575444 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session c351a89a5fe34e788b9d8064fbffa751-184c03808a1e470dba9c6bfa8e04774a on tablet 184c03808a1e470dba9c6bfa8e04774a with peer c351a89a5fe34e788b9d8064fbffa751
I20240502 14:00:56.592103 6600 tablet_bootstrap.cc:492] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:56.592411 6600 tablet_bootstrap.cc:492] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:56.592607 6600 ts_tablet_manager.cc:1367] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.032s user 0.026s sys 0.000s
I20240502 14:00:56.593080 6600 raft_consensus.cc:348] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:56.593195 6600 raft_consensus.cc:729] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:56.593299 6600 consensus_queue.cc:260] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:56.593617 6600 ts_tablet_manager.cc:1398] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:56.595050 562 tablet_copy_service.cc:342] P c351a89a5fe34e788b9d8064fbffa751: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-8568aa57fce64ccf89a36deb5b7b0f3b received from {username='slave'} at 127.26.25.135:53809
I20240502 14:00:56.595124 562 tablet_copy_service.cc:433] P c351a89a5fe34e788b9d8064fbffa751: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-8568aa57fce64ccf89a36deb5b7b0f3b on tablet 8568aa57fce64ccf89a36deb5b7b0f3b with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:56.652154 2886 raft_consensus.cc:1203] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:56.672386 805 consensus_queue.cc:237] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
W20240502 14:00:56.673908 738 consensus_peers.cc:485] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 -> Peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): Couldn't send request to peer 452ecbc085e54db49c3b01c20a96f860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85ad896b8838472096851682071c0a9c. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:56.673940 673 raft_consensus.cc:1261] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:56.674474 405 raft_consensus.cc:1261] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:56.674818 6174 consensus_queue.cc:1035] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.676564 6174 consensus_queue.cc:1035] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.677613 6174 raft_consensus.cc:2932] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:56.679890 405 raft_consensus.cc:2932] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:56.681125 32732 catalog_manager.cc:5484] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:56.684693 673 raft_consensus.cc:2932] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:56.696995 805 raft_consensus.cc:1203] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:56.702207 540 raft_consensus.cc:1203] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:56.706435 673 consensus_queue.cc:237] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:56.707608 3149 raft_consensus.cc:1203] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20240502 14:00:56.707999 607 consensus_peers.cc:485] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e9f86a6149d43f3a217769865d9a3d9. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:56.708050 805 raft_consensus.cc:1261] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:56.708482 540 raft_consensus.cc:1261] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:56.708966 6065 consensus_queue.cc:1035] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.709126 6221 consensus_queue.cc:1035] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.710850 6065 raft_consensus.cc:2932] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:56.711340 540 raft_consensus.cc:2932] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:56.716255 32732 catalog_manager.cc:5484] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:56.718502 805 raft_consensus.cc:2932] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:56.725056 405 raft_consensus.cc:1203] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:56.765731 3019 consensus_queue.cc:237] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:56.767041 3284 raft_consensus.cc:1261] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:00:56.767118 805 raft_consensus.cc:1261] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:00:56.767889 6068 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:56.768389 6068 consensus_queue.cc:1035] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:56.770556 6202 raft_consensus.cc:2932] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:56.770982 805 raft_consensus.cc:2932] T 909bc2eb63544db0b4b3a0764be7d4a1 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:56.772838 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 909bc2eb63544db0b4b3a0764be7d4a1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:56.774536 32731 catalog_manager.cc:5484] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 3 to 5, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 2 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:56.775147 3284 raft_consensus.cc:2932] T 909bc2eb63544db0b4b3a0764be7d4a1 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:56.780555 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 909bc2eb63544db0b4b3a0764be7d4a1 with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:56.781302 3284 raft_consensus.cc:1203] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:56.787415 6984 tablet_replica.cc:330] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:56.787567 6984 raft_consensus.cc:2229] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:56.787683 6984 raft_consensus.cc:2258] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:56.788304 6984 ts_tablet_manager.cc:1875] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:56.790365 6984 ts_tablet_manager.cc:1888] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20240502 14:00:56.790453 6984 log.cc:1196] T 909bc2eb63544db0b4b3a0764be7d4a1 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/909bc2eb63544db0b4b3a0764be7d4a1
I20240502 14:00:56.791007 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 909bc2eb63544db0b4b3a0764be7d4a1 (table rebalancer_extra_table_16 [id=0b8348df8f78415b994322df0d6d1090]) successfully deleted
I20240502 14:00:56.794669 805 consensus_queue.cc:237] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:56.796044 3019 raft_consensus.cc:1261] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:00:56.796073 405 raft_consensus.cc:1261] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:00:56.796473 6799 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:56.796698 6934 consensus_queue.cc:1035] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:56.798522 6799 raft_consensus.cc:2932] T 094aa25b08b8498f9ae4493cafd289e3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:56.799141 3019 raft_consensus.cc:2932] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:56.800395 405 raft_consensus.cc:2932] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:56.801321 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 094aa25b08b8498f9ae4493cafd289e3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:56.803650 32731 catalog_manager.cc:5484] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 3 to 5, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New cstate: current_term: 2 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:56.808291 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet 094aa25b08b8498f9ae4493cafd289e3 with delete_type TABLET_DATA_TOMBSTONED (TS da3f8160557e454f8547f45e80c2f013 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:56.808683 6995 tablet_replica.cc:330] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:56.810236 6995 raft_consensus.cc:2229] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:56.810366 6995 raft_consensus.cc:2258] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:56.810959 6995 ts_tablet_manager.cc:1875] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:56.814982 6995 ts_tablet_manager.cc:1888] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20240502 14:00:56.815889 6995 log.cc:1196] T 094aa25b08b8498f9ae4493cafd289e3 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/094aa25b08b8498f9ae4493cafd289e3
I20240502 14:00:56.816661 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 094aa25b08b8498f9ae4493cafd289e3 (table rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]) successfully deleted
I20240502 14:00:56.819701 2886 raft_consensus.cc:1203] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:56.887094 673 consensus_queue.cc:237] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
W20240502 14:00:56.888653 607 consensus_peers.cc:485] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: edcd3af1a04c43d3ab79d539b758ecb0. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:56.889168 405 raft_consensus.cc:1261] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:56.889533 540 raft_consensus.cc:1261] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:56.889878 6221 consensus_queue.cc:1035] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.890678 6221 consensus_queue.cc:1035] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.892210 5098 raft_consensus.cc:2932] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:56.892617 405 raft_consensus.cc:2932] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:56.894296 541 raft_consensus.cc:2932] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:56.896869 32732 catalog_manager.cc:5484] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:56.914597 5282 raft_consensus.cc:1050] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860: attempting to promote NON_VOTER c351a89a5fe34e788b9d8064fbffa751 to VOTER
I20240502 14:00:56.915081 5282 consensus_queue.cc:237] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:56.916391 541 raft_consensus.cc:1261] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:56.916445 3284 raft_consensus.cc:1261] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:56.916779 803 raft_consensus.cc:1261] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:56.916870 6068 consensus_queue.cc:1035] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:56.917394 6202 consensus_queue.cc:1035] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:56.918121 6202 consensus_queue.cc:1035] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:56.920023 6068 raft_consensus.cc:2932] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:56.920598 803 raft_consensus.cc:2932] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:56.921972 541 raft_consensus.cc:2932] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:56.923851 3284 raft_consensus.cc:2932] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:56.925649 32734 catalog_manager.cc:5484] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:56.932852 3019 consensus_queue.cc:237] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:56.933944 540 raft_consensus.cc:1261] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:56.934621 3281 raft_consensus.cc:1261] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:56.935119 5282 consensus_queue.cc:1035] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.935525 5282 consensus_queue.cc:1035] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:56.936928 6068 raft_consensus.cc:2932] T 792bc86bad314a2bbf4bba3001c78894 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:56.937472 3281 raft_consensus.cc:2932] T 792bc86bad314a2bbf4bba3001c78894 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:56.937563 540 raft_consensus.cc:2932] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:56.939666 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 792bc86bad314a2bbf4bba3001c78894 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:56.941114 32731 catalog_manager.cc:5484] T 792bc86bad314a2bbf4bba3001c78894 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:56.944176 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 792bc86bad314a2bbf4bba3001c78894 with delete_type TABLET_DATA_TOMBSTONED (TS 478dfa53d7944f38b4e6cabee21b2d01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:56.945554 7084 tablet_replica.cc:330] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:56.945737 7084 raft_consensus.cc:2229] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:56.945894 7084 raft_consensus.cc:2258] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:56.946564 7084 ts_tablet_manager.cc:1875] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:56.948992 7084 ts_tablet_manager.cc:1888] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:56.949112 7084 log.cc:1196] T 792bc86bad314a2bbf4bba3001c78894 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/792bc86bad314a2bbf4bba3001c78894
I20240502 14:00:56.949802 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 792bc86bad314a2bbf4bba3001c78894 (table rebalancer_extra_table_62 [id=bb7573d3e8ad494cb5dd23a0f882463f]) successfully deleted
I20240502 14:00:56.964010 3150 raft_consensus.cc:1203] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:56.991865 5098 raft_consensus.cc:1050] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER da3f8160557e454f8547f45e80c2f013 to VOTER
I20240502 14:00:56.992367 5098 consensus_queue.cc:237] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:56.993755 540 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:56.993777 3281 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:56.993794 803 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:56.994215 5711 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:56.994378 6221 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:56.994864 6065 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:56.998925 6065 raft_consensus.cc:2932] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.000339 3281 raft_consensus.cc:2932] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.000908 540 raft_consensus.cc:2932] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.001736 803 raft_consensus.cc:2932] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.004832 32732 catalog_manager.cc:5484] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.083710 6840 raft_consensus.cc:1050] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:00:57.084198 6840 consensus_queue.cc:237] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:57.085448 3149 raft_consensus.cc:1261] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.085484 2884 raft_consensus.cc:1261] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.085943 3019 raft_consensus.cc:1261] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.086047 540 consensus_queue.cc:237] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:57.086323 7166 consensus_queue.cc:1035] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.087119 5715 consensus_queue.cc:1035] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.087431 5715 consensus_queue.cc:1035] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.087803 3149 raft_consensus.cc:1261] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:57.087874 477 consensus_peers.cc:485] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: db0bf9f6821045e1a259703fc3445226. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.088532 6834 consensus_queue.cc:1035] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.088765 2885 raft_consensus.cc:1261] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.089290 6834 consensus_queue.cc:1035] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.089376 5715 raft_consensus.cc:1050] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e: attempting to promote NON_VOTER 452ecbc085e54db49c3b01c20a96f860 to VOTER
I20240502 14:00:57.089758 5715 consensus_queue.cc:237] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:00:57.090394 3150 raft_consensus.cc:2932] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.090698 5715 raft_consensus.cc:2932] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.091132 6834 raft_consensus.cc:2932] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.092478 2885 raft_consensus.cc:2932] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.093537 2884 raft_consensus.cc:2932] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.094878 32731 catalog_manager.cc:5484] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.095604 673 raft_consensus.cc:1261] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.096392 32728 catalog_manager.cc:5484] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.097129 6171 consensus_queue.cc:1035] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.097837 540 raft_consensus.cc:1261] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.098403 6171 consensus_queue.cc:1035] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.099313 3019 raft_consensus.cc:1261] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.100747 5655 raft_consensus.cc:1050] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:00:57.101168 5655 consensus_queue.cc:237] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:57.101452 6840 consensus_queue.cc:1035] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.102033 3019 raft_consensus.cc:2932] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.104358 803 raft_consensus.cc:1261] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.104760 5655 consensus_queue.cc:1035] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.104846 405 raft_consensus.cc:1261] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.105171 2885 raft_consensus.cc:1261] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.105448 5711 raft_consensus.cc:1050] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 to VOTER
I20240502 14:00:57.105831 5711 consensus_queue.cc:237] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:00:57.107606 7181 consensus_queue.cc:1035] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.107679 805 raft_consensus.cc:1261] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.108114 6221 consensus_queue.cc:1035] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.108168 2885 raft_consensus.cc:1261] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.108690 6221 consensus_queue.cc:1035] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.109987 541 raft_consensus.cc:1261] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.110392 6221 consensus_queue.cc:1035] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.110564 5655 consensus_queue.cc:1035] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.111366 3150 raft_consensus.cc:2932] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.112685 405 consensus_queue.cc:237] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:57.116564 6840 raft_consensus.cc:2932] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:57.116786 3150 raft_consensus.cc:1261] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.117101 2886 raft_consensus.cc:1261] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.117174 5715 consensus_queue.cc:1035] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.117492 5715 consensus_queue.cc:1035] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.118860 6171 raft_consensus.cc:2932] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.118872 3017 raft_consensus.cc:2932] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:57.118959 541 raft_consensus.cc:2932] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:57.120582 32720 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet caffa941c89c48d1b3d36ca47dc49eb2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.121151 32731 catalog_manager.cc:5484] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.121759 2886 raft_consensus.cc:2932] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.122504 673 raft_consensus.cc:2932] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:57.124040 6221 raft_consensus.cc:2932] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.124191 3150 raft_consensus.cc:2932] T caffa941c89c48d1b3d36ca47dc49eb2 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.126555 803 raft_consensus.cc:2932] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.126871 540 raft_consensus.cc:2932] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.126945 32734 catalog_manager.cc:5484] T caffa941c89c48d1b3d36ca47dc49eb2 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.127946 32726 catalog_manager.cc:5484] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:57.129992 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet caffa941c89c48d1b3d36ca47dc49eb2 with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:57.131877 5655 raft_consensus.cc:2932] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.132305 2884 raft_consensus.cc:2932] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.133839 409 raft_consensus.cc:2932] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.137023 803 raft_consensus.cc:2932] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.137995 32726 catalog_manager.cc:5484] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.138109 2885 raft_consensus.cc:2932] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.140465 7195 tablet_replica.cc:330] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:57.140650 7195 raft_consensus.cc:2229] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.140671 5282 raft_consensus.cc:1050] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:00:57.140807 7195 raft_consensus.cc:2258] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.141089 5282 consensus_queue.cc:237] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:57.141444 7195 ts_tablet_manager.cc:1875] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.142529 2885 raft_consensus.cc:1261] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.142880 3284 raft_consensus.cc:1261] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.142884 409 raft_consensus.cc:1261] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.143251 7071 consensus_queue.cc:1035] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.144259 6068 consensus_queue.cc:1035] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.144578 6068 consensus_queue.cc:1035] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.144795 7195 ts_tablet_manager.cc:1888] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:57.144858 7195 log.cc:1196] T caffa941c89c48d1b3d36ca47dc49eb2 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/caffa941c89c48d1b3d36ca47dc49eb2
I20240502 14:00:57.145947 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet caffa941c89c48d1b3d36ca47dc49eb2 (table rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]) successfully deleted
I20240502 14:00:57.147559 6202 raft_consensus.cc:2932] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.148027 409 raft_consensus.cc:2932] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.149391 3284 raft_consensus.cc:2932] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.149617 3150 raft_consensus.cc:1203] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:57.150014 2885 raft_consensus.cc:2932] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.151262 673 consensus_queue.cc:237] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:57.153028 409 consensus_queue.cc:237] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:00:57.153923 2885 raft_consensus.cc:1261] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.155604 541 raft_consensus.cc:1203] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:57.156239 3019 raft_consensus.cc:1261] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:57.156628 5098 consensus_queue.cc:1035] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:57.157131 606 consensus_peers.cc:485] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 618c1b9af9504c03bd250d88c3a2b0ba. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.157227 7174 consensus_queue.cc:1035] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.158246 541 raft_consensus.cc:1261] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:57.158404 32731 catalog_manager.cc:5484] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.159029 7174 consensus_queue.cc:1035] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.160270 3284 raft_consensus.cc:1261] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.160737 5098 consensus_queue.cc:1035] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.162781 672 consensus_queue.cc:237] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:00:57.163319 3284 raft_consensus.cc:2932] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:57.164245 2884 raft_consensus.cc:1261] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:57.164393 540 consensus_queue.cc:237] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:57.164575 6065 consensus_queue.cc:1035] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.165732 804 raft_consensus.cc:1261] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.165748 405 raft_consensus.cc:1261] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.166132 7181 consensus_queue.cc:1035] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.166805 6939 raft_consensus.cc:1050] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:00:57.167189 6939 consensus_queue.cc:237] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:57.168045 540 raft_consensus.cc:1261] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:57.162709 5098 raft_consensus.cc:2932] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:57.168501 5711 consensus_queue.cc:1035] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.169252 6834 consensus_queue.cc:1035] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.169304 3017 raft_consensus.cc:1261] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.169675 7174 raft_consensus.cc:2932] T 1bdc7b131596496aa6779885fe5d9433 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:57.170904 540 raft_consensus.cc:1261] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.171424 5674 consensus_queue.cc:1035] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.171525 6065 raft_consensus.cc:2932] T be795fbe277b4e9fac8561d6589b7c05 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.171855 540 raft_consensus.cc:2932] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.171972 32720 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1bdc7b131596496aa6779885fe5d9433 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.173487 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet be795fbe277b4e9fac8561d6589b7c05 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.174149 3019 consensus_queue.cc:237] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:57.174571 3017 raft_consensus.cc:2932] T 1bdc7b131596496aa6779885fe5d9433 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:57.175469 540 raft_consensus.cc:2932] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:57.176360 3284 raft_consensus.cc:1261] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.176594 5674 consensus_queue.cc:1035] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.176882 2886 raft_consensus.cc:2932] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:57.177793 32726 catalog_manager.cc:5484] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:57.179957 32729 catalog_manager.cc:5484] T be795fbe277b4e9fac8561d6589b7c05 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.179949 2884 raft_consensus.cc:2932] T be795fbe277b4e9fac8561d6589b7c05 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.180238 32729 catalog_manager.cc:5484] T 1bdc7b131596496aa6779885fe5d9433 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:57.180920 5655 raft_consensus.cc:2932] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.181322 5282 consensus_queue.cc:1035] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.182523 805 raft_consensus.cc:2932] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.183625 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ad4e1b6dea1e4beeb4b82852ded83c5f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.184302 405 raft_consensus.cc:1261] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.186167 5282 consensus_queue.cc:1035] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.187537 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdc7b131596496aa6779885fe5d9433 with delete_type TABLET_DATA_TOMBSTONED (TS 58107b611f8f475a910a6fd54dfd1fb9 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:57.187970 785 tablet_service.cc:1514] Processing DeleteTablet for tablet be795fbe277b4e9fac8561d6589b7c05 with delete_type TABLET_DATA_TOMBSTONED (TS 478dfa53d7944f38b4e6cabee21b2d01 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:57.188170 7084 tablet_replica.cc:330] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:57.188318 7084 raft_consensus.cc:2229] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.188478 7084 raft_consensus.cc:2258] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.188449 5282 raft_consensus.cc:2932] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.189283 5098 raft_consensus.cc:1050] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:00:57.189770 5098 consensus_queue.cc:237] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:57.190107 803 raft_consensus.cc:1261] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.190382 7084 ts_tablet_manager.cc:1875] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.191342 409 raft_consensus.cc:1261] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.191429 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e93a3a3fca9042e9b8bdf3c60ec73a20 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.191543 3150 raft_consensus.cc:1261] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.191746 6221 consensus_queue.cc:1035] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.191947 5098 consensus_queue.cc:1035] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.192189 7175 raft_consensus.cc:1050] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e: attempting to promote NON_VOTER c351a89a5fe34e788b9d8064fbffa751 to VOTER
I20240502 14:00:57.192637 7084 ts_tablet_manager.cc:1888] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:57.192723 7084 log.cc:1196] T be795fbe277b4e9fac8561d6589b7c05 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/be795fbe277b4e9fac8561d6589b7c05
I20240502 14:00:57.192644 7175 consensus_queue.cc:237] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:57.193346 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet be795fbe277b4e9fac8561d6589b7c05 (table rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]) successfully deleted
I20240502 14:00:57.193768 7206 consensus_queue.cc:1035] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.194447 3149 raft_consensus.cc:1261] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.194381 32726 catalog_manager.cc:5484] T ad4e1b6dea1e4beeb4b82852ded83c5f P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.195060 5715 consensus_queue.cc:1035] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.195791 405 raft_consensus.cc:2932] T ad4e1b6dea1e4beeb4b82852ded83c5f P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.197185 540 raft_consensus.cc:1261] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.197595 805 raft_consensus.cc:1261] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.198379 7210 tablet_replica.cc:330] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:57.198523 7210 raft_consensus.cc:2229] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.198691 7210 raft_consensus.cc:2258] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.199967 540 raft_consensus.cc:1261] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.200389 6840 consensus_queue.cc:1035] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.200821 32734 catalog_manager.cc:5484] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.201108 405 raft_consensus.cc:2932] T e93a3a3fca9042e9b8bdf3c60ec73a20 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.203372 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet ad4e1b6dea1e4beeb4b82852ded83c5f with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:57.203251 3284 raft_consensus.cc:2932] T e93a3a3fca9042e9b8bdf3c60ec73a20 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:57.203612 6984 tablet_replica.cc:330] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:57.203780 6984 raft_consensus.cc:2229] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.203967 6984 raft_consensus.cc:2258] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.204054 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet e93a3a3fca9042e9b8bdf3c60ec73a20 with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:57.205013 7210 ts_tablet_manager.cc:1875] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.207953 7210 ts_tablet_manager.cc:1888] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:57.208029 7210 log.cc:1196] T 1bdc7b131596496aa6779885fe5d9433 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/1bdc7b131596496aa6779885fe5d9433
I20240502 14:00:57.208698 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 1bdc7b131596496aa6779885fe5d9433 (table rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]) successfully deleted
I20240502 14:00:57.208849 7174 consensus_queue.cc:1035] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.209100 5711 raft_consensus.cc:2932] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.209513 6221 consensus_queue.cc:1035] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.211238 542 raft_consensus.cc:2932] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.211499 405 raft_consensus.cc:2932] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.211606 3149 raft_consensus.cc:2932] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.215926 32729 catalog_manager.cc:5484] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.216946 7202 ts_tablet_manager.cc:902] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860: Initiating tablet copy from peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:00:57.217303 7174 raft_consensus.cc:2932] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.217541 7202 tablet_copy_client.cc:308] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.132:33443
I20240502 14:00:57.219055 7200 ts_tablet_manager.cc:902] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:57.219533 541 raft_consensus.cc:2932] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.219597 3152 raft_consensus.cc:2932] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.223395 7200 tablet_copy_client.cc:308] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:57.224210 32728 catalog_manager.cc:5484] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.224335 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 7e9f86a6149d43f3a217769865d9a3d9 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:47487)
I20240502 14:00:57.224433 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 7e9f86a6149d43f3a217769865d9a3d9 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:47487: session id = da3f8160557e454f8547f45e80c2f013-7e9f86a6149d43f3a217769865d9a3d9
I20240502 14:00:57.225030 6174 raft_consensus.cc:1050] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01: attempting to promote NON_VOTER da3f8160557e454f8547f45e80c2f013 to VOTER
I20240502 14:00:57.225466 693 tablet_copy_source_session.cc:205] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:57.225468 6174 consensus_queue.cc:237] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:57.226815 409 raft_consensus.cc:1261] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.226842 672 raft_consensus.cc:1261] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.227197 6799 consensus_queue.cc:1035] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.227459 7200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e9f86a6149d43f3a217769865d9a3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.227535 6799 consensus_queue.cc:1035] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.230100 826 tablet_copy_service.cc:140] P 478dfa53d7944f38b4e6cabee21b2d01: Received BeginTabletCopySession request for tablet 85ad896b8838472096851682071c0a9c from peer 452ecbc085e54db49c3b01c20a96f860 ({username='slave'} at 127.26.25.134:46199)
I20240502 14:00:57.230192 826 tablet_copy_service.cc:161] P 478dfa53d7944f38b4e6cabee21b2d01: Beginning new tablet copy session on tablet 85ad896b8838472096851682071c0a9c from peer 452ecbc085e54db49c3b01c20a96f860 at {username='slave'} at 127.26.25.134:46199: session id = 452ecbc085e54db49c3b01c20a96f860-85ad896b8838472096851682071c0a9c
I20240502 14:00:57.230280 7200 tablet_copy_client.cc:783] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:57.230443 7200 tablet_copy_client.cc:647] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:57.231221 826 tablet_copy_source_session.cc:205] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:57.232360 3281 raft_consensus.cc:1261] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.233595 6174 consensus_queue.cc:1035] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.233644 7202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85ad896b8838472096851682071c0a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.234191 7200 tablet_copy_client.cc:523] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:57.236631 409 consensus_queue.cc:237] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:57.236805 7202 tablet_copy_client.cc:783] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:57.236966 7202 tablet_copy_client.cc:647] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:57.237844 7200 tablet_bootstrap.cc:492] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:57.237847 542 raft_consensus.cc:1261] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.238526 6984 ts_tablet_manager.cc:1875] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.238462 5674 raft_consensus.cc:2932] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:57.238584 7175 consensus_queue.cc:1035] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.240643 7175 raft_consensus.cc:2932] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.242065 3017 raft_consensus.cc:2932] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:57.242408 32720 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a176c4678de94ccc93b14d04fc46ea90 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.242799 32726 catalog_manager.cc:5484] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.242908 541 raft_consensus.cc:2932] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.243534 6799 raft_consensus.cc:2932] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.244108 803 raft_consensus.cc:2932] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:57.245623 3149 raft_consensus.cc:1261] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.246166 7166 consensus_queue.cc:1035] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.246856 542 raft_consensus.cc:2932] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:57.247414 32728 catalog_manager.cc:5484] T a176c4678de94ccc93b14d04fc46ea90 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.247738 6984 ts_tablet_manager.cc:1888] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:57.247807 6984 log.cc:1196] T ad4e1b6dea1e4beeb4b82852ded83c5f P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/ad4e1b6dea1e4beeb4b82852ded83c5f
I20240502 14:00:57.248260 6984 tablet_replica.cc:330] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:57.248389 6984 raft_consensus.cc:2229] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.248507 6984 raft_consensus.cc:2258] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.248603 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet ad4e1b6dea1e4beeb4b82852ded83c5f (table rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]) successfully deleted
I20240502 14:00:57.249667 6984 ts_tablet_manager.cc:1875] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.250769 32726 catalog_manager.cc:5484] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:57.251562 672 raft_consensus.cc:2932] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.252666 405 raft_consensus.cc:2932] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.254793 3149 raft_consensus.cc:2932] T a176c4678de94ccc93b14d04fc46ea90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.255993 785 tablet_service.cc:1514] Processing DeleteTablet for tablet a176c4678de94ccc93b14d04fc46ea90 with delete_type TABLET_DATA_TOMBSTONED (TS 478dfa53d7944f38b4e6cabee21b2d01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:57.256215 7084 tablet_replica.cc:330] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:57.256351 7084 raft_consensus.cc:2229] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.256458 7084 pending_rounds.cc:62] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01: Trying to abort 1 pending ops.
I20240502 14:00:57.256525 7084 pending_rounds.cc:68] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7023241040659914752 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "a176c4678de94ccc93b14d04fc46ea90" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } } }
I20240502 14:00:57.256678 7084 raft_consensus.cc:2894] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20240502 14:00:57.256749 7084 raft_consensus.cc:2866] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 14:00:57.256801 7084 raft_consensus.cc:2258] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.259296 2884 consensus_queue.cc:237] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:57.260957 3017 raft_consensus.cc:1261] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.260952 803 raft_consensus.cc:1261] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.261936 5674 consensus_queue.cc:1035] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.262423 7207 consensus_queue.cc:1035] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.264117 7202 tablet_copy_client.cc:523] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:57.264737 7207 raft_consensus.cc:2932] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:57.265604 803 raft_consensus.cc:2932] T 58a0cf9cd92a43ca8c481964707eda6d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:57.266135 3019 raft_consensus.cc:2932] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:57.268239 7084 ts_tablet_manager.cc:1875] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.272069 7202 tablet_bootstrap.cc:492] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:57.274498 3281 raft_consensus.cc:2932] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.275028 32726 catalog_manager.cc:5484] T 58a0cf9cd92a43ca8c481964707eda6d P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:57.279673 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 58a0cf9cd92a43ca8c481964707eda6d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.282755 6984 ts_tablet_manager.cc:1888] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:57.282855 6984 log.cc:1196] T e93a3a3fca9042e9b8bdf3c60ec73a20 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/e93a3a3fca9042e9b8bdf3c60ec73a20
I20240502 14:00:57.283385 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet e93a3a3fca9042e9b8bdf3c60ec73a20 (table rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]) successfully deleted
I20240502 14:00:57.283696 7084 ts_tablet_manager.cc:1888] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:57.283792 7084 log.cc:1196] T a176c4678de94ccc93b14d04fc46ea90 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/a176c4678de94ccc93b14d04fc46ea90
I20240502 14:00:57.284490 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet a176c4678de94ccc93b14d04fc46ea90 (table rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]) successfully deleted
I20240502 14:00:57.285537 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 58a0cf9cd92a43ca8c481964707eda6d with delete_type TABLET_DATA_TOMBSTONED (TS c351a89a5fe34e788b9d8064fbffa751 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:57.290397 803 consensus_queue.cc:237] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:57.292951 409 raft_consensus.cc:1261] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:57.293090 3281 raft_consensus.cc:1261] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:57.293432 6799 consensus_queue.cc:1035] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.293612 6934 consensus_queue.cc:1035] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.295285 6934 raft_consensus.cc:2932] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.296211 3281 raft_consensus.cc:2932] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.297083 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9486274cd39b4de18c37ab9f71f261c2 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.300343 32729 catalog_manager.cc:5484] T 9486274cd39b4de18c37ab9f71f261c2 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.300714 7238 tablet_replica.cc:330] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:57.300864 7238 raft_consensus.cc:2229] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.301184 405 raft_consensus.cc:2932] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:57.302601 7238 raft_consensus.cc:2258] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.303287 7238 ts_tablet_manager.cc:1875] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.304096 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 9486274cd39b4de18c37ab9f71f261c2 with delete_type TABLET_DATA_TOMBSTONED (TS 58107b611f8f475a910a6fd54dfd1fb9 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:57.304307 7210 tablet_replica.cc:330] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:57.304462 7210 raft_consensus.cc:2229] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.304625 7210 raft_consensus.cc:2258] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.305938 6625 raft_consensus.cc:1050] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485: attempting to promote NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 to VOTER
I20240502 14:00:57.306360 6625 consensus_queue.cc:237] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:00:57.308143 3281 raft_consensus.cc:1261] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.308346 2884 raft_consensus.cc:1261] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.308506 7006 consensus_queue.cc:1035] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.308702 6066 consensus_queue.cc:1035] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.310180 405 raft_consensus.cc:1261] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.310561 7006 consensus_queue.cc:1035] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.312358 7200 tablet_bootstrap.cc:492] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:57.312702 7210 ts_tablet_manager.cc:1875] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.312978 7006 raft_consensus.cc:2932] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.313755 7238 ts_tablet_manager.cc:1888] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:57.313828 7238 log.cc:1196] T 58a0cf9cd92a43ca8c481964707eda6d P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/58a0cf9cd92a43ca8c481964707eda6d
I20240502 14:00:57.314487 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 58a0cf9cd92a43ca8c481964707eda6d (table rebalancer_extra_table_18 [id=c2a7fffdc4d148a5839276762398bfd9]) successfully deleted
I20240502 14:00:57.315089 7210 ts_tablet_manager.cc:1888] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:57.315153 7210 log.cc:1196] T 9486274cd39b4de18c37ab9f71f261c2 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/9486274cd39b4de18c37ab9f71f261c2
I20240502 14:00:57.315354 7200 tablet_bootstrap.cc:492] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:00:57.315549 7200 ts_tablet_manager.cc:1367] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.078s user 0.024s sys 0.004s
I20240502 14:00:57.315625 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 9486274cd39b4de18c37ab9f71f261c2 (table rebalance_test_table_15 [id=d4238514953942f2a3e7af65ce92b025]) successfully deleted
I20240502 14:00:57.316560 7200 raft_consensus.cc:348] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:57.316484 405 raft_consensus.cc:2932] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.316685 7200 raft_consensus.cc:729] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:00:57.316782 7200 consensus_queue.cc:260] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:57.317109 7200 ts_tablet_manager.cc:1398] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:57.318372 3284 raft_consensus.cc:2932] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.318562 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-7e9f86a6149d43f3a217769865d9a3d9 received from {username='slave'} at 127.26.25.136:47487
I20240502 14:00:57.318629 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session da3f8160557e454f8547f45e80c2f013-7e9f86a6149d43f3a217769865d9a3d9 on tablet 7e9f86a6149d43f3a217769865d9a3d9 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:00:57.322743 32728 catalog_manager.cc:5484] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 2 to 3, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.322796 2884 raft_consensus.cc:2932] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.325831 7206 raft_consensus.cc:1050] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:00:57.326236 7206 consensus_queue.cc:237] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:57.327908 3149 raft_consensus.cc:1261] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.328130 541 raft_consensus.cc:1261] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.328130 3019 raft_consensus.cc:1261] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.328338 6939 consensus_queue.cc:1035] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.328701 6939 consensus_queue.cc:1035] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.329021 6939 consensus_queue.cc:1035] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.332960 5674 raft_consensus.cc:2932] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.335635 3019 raft_consensus.cc:2932] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.336149 7202 tablet_bootstrap.cc:492] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:57.336491 7202 tablet_bootstrap.cc:492] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860: Bootstrap complete.
I20240502 14:00:57.336472 541 raft_consensus.cc:2932] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.336692 7202 ts_tablet_manager.cc:1367] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.065s user 0.027s sys 0.000s
I20240502 14:00:57.337630 7202 raft_consensus.cc:348] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:57.337710 7202 raft_consensus.cc:729] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: LEARNER
I20240502 14:00:57.337623 3150 consensus_queue.cc:237] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:00:57.337788 7202 consensus_queue.cc:260] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:57.338177 7202 ts_tablet_manager.cc:1398] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:57.339148 3284 raft_consensus.cc:1261] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.339282 3149 raft_consensus.cc:2932] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.340030 32728 catalog_manager.cc:5484] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.340924 6625 consensus_queue.cc:1035] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.342190 2884 raft_consensus.cc:1261] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.342531 826 tablet_copy_service.cc:342] P 478dfa53d7944f38b4e6cabee21b2d01: Request end of tablet copy session 452ecbc085e54db49c3b01c20a96f860-85ad896b8838472096851682071c0a9c received from {username='slave'} at 127.26.25.134:46199
I20240502 14:00:57.342624 826 tablet_copy_service.cc:433] P 478dfa53d7944f38b4e6cabee21b2d01: ending tablet copy session 452ecbc085e54db49c3b01c20a96f860-85ad896b8838472096851682071c0a9c on tablet 85ad896b8838472096851682071c0a9c with peer 452ecbc085e54db49c3b01c20a96f860
I20240502 14:00:57.343894 7006 consensus_queue.cc:1035] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.345669 7246 raft_consensus.cc:2932] T 564ba4de769e45af8de10fe8c86eef31 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.346350 2884 consensus_queue.cc:237] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:57.347735 541 raft_consensus.cc:1261] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.347806 3281 raft_consensus.cc:2932] T 564ba4de769e45af8de10fe8c86eef31 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.348497 6939 consensus_queue.cc:1035] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.348995 2884 raft_consensus.cc:2932] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.351083 3150 raft_consensus.cc:1261] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.351603 7206 consensus_queue.cc:1035] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.353106 32729 catalog_manager.cc:5484] T 564ba4de769e45af8de10fe8c86eef31 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:00:57.354519 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 564ba4de769e45af8de10fe8c86eef31 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.357422 387 tablet_service.cc:1514] Processing DeleteTablet for tablet 564ba4de769e45af8de10fe8c86eef31 with delete_type TABLET_DATA_TOMBSTONED (TS f003b732dd0f46958262b42f8d20305e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:57.358750 7259 tablet_replica.cc:330] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:57.358996 7259 raft_consensus.cc:2229] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.359243 7259 raft_consensus.cc:2258] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.359385 7206 raft_consensus.cc:2932] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.360095 7259 ts_tablet_manager.cc:1875] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.361932 541 raft_consensus.cc:2932] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.362349 7259 ts_tablet_manager.cc:1888] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:57.362414 7259 log.cc:1196] T 564ba4de769e45af8de10fe8c86eef31 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/564ba4de769e45af8de10fe8c86eef31
I20240502 14:00:57.362465 3150 raft_consensus.cc:2932] T 10d6cc99575f4ebb8c980e9cb3e35145 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.363615 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 564ba4de769e45af8de10fe8c86eef31 (table rebalancer_extra_table_46 [id=70b6fe08fece4cc7b481dfd7c0360f3e]) successfully deleted
I20240502 14:00:57.366336 32728 catalog_manager.cc:5484] T 10d6cc99575f4ebb8c980e9cb3e35145 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.366987 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 10d6cc99575f4ebb8c980e9cb3e35145 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.370745 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 10d6cc99575f4ebb8c980e9cb3e35145 with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:57.371042 7195 tablet_replica.cc:330] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:57.371208 7195 raft_consensus.cc:2229] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.371358 7195 raft_consensus.cc:2258] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.372000 7195 ts_tablet_manager.cc:1875] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.373912 7195 ts_tablet_manager.cc:1888] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:57.373982 7195 log.cc:1196] T 10d6cc99575f4ebb8c980e9cb3e35145 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/10d6cc99575f4ebb8c980e9cb3e35145
I20240502 14:00:57.374490 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 10d6cc99575f4ebb8c980e9cb3e35145 (table rebalancer_extra_table_72 [id=c4ee0314ba8244b5aa832bb31e2b0534]) successfully deleted
I20240502 14:00:57.423525 672 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80"
dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:57520
I20240502 14:00:57.423684 672 raft_consensus.cc:595] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:57.433513 7200 ts_tablet_manager.cc:902] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:57.433985 7200 tablet_copy_client.cc:308] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:57.434804 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet edcd3af1a04c43d3ab79d539b758ecb0 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:47487)
I20240502 14:00:57.434922 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet edcd3af1a04c43d3ab79d539b758ecb0 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:47487: session id = da3f8160557e454f8547f45e80c2f013-edcd3af1a04c43d3ab79d539b758ecb0
I20240502 14:00:57.435945 693 tablet_copy_source_session.cc:205] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:57.437024 7200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edcd3af1a04c43d3ab79d539b758ecb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.439469 7200 tablet_copy_client.cc:783] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:57.439625 7200 tablet_copy_client.cc:647] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:57.441848 7200 tablet_copy_client.cc:523] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:57.443957 7200 tablet_bootstrap.cc:492] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:57.469038 7200 tablet_bootstrap.cc:492] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:57.469334 7200 tablet_bootstrap.cc:492] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:00:57.469516 7200 ts_tablet_manager.cc:1367] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.026s user 0.021s sys 0.000s
I20240502 14:00:57.469954 7200 raft_consensus.cc:348] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:57.470079 7200 raft_consensus.cc:729] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:00:57.470167 7200 consensus_queue.cc:260] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:57.470433 7200 ts_tablet_manager.cc:1398] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:57.471597 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-edcd3af1a04c43d3ab79d539b758ecb0 received from {username='slave'} at 127.26.25.136:47487
I20240502 14:00:57.471673 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session da3f8160557e454f8547f45e80c2f013-edcd3af1a04c43d3ab79d539b758ecb0 on tablet edcd3af1a04c43d3ab79d539b758ecb0 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:00:57.474521 6221 raft_consensus.cc:979] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: : Instructing follower 478dfa53d7944f38b4e6cabee21b2d01 to start an election
I20240502 14:00:57.474620 6221 raft_consensus.cc:1067] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Signalling peer 478dfa53d7944f38b4e6cabee21b2d01 to start an election
I20240502 14:00:57.475666 804 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80"
dest_uuid: "478dfa53d7944f38b4e6cabee21b2d01"
from {username='slave'} at 127.26.25.131:40729
I20240502 14:00:57.475783 804 raft_consensus.cc:482] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:57.475870 804 raft_consensus.cc:3037] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:57.477010 804 raft_consensus.cc:504] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:57.477492 804 leader_election.cc:288] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 2 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:57.478082 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:00:57.478215 541 raft_consensus.cc:3037] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:57.478343 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:00:57.478397 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80" candidate_uuid: "478dfa53d7944f38b4e6cabee21b2d01" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:00:57.478448 3284 raft_consensus.cc:3037] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:57.478509 672 raft_consensus.cc:3032] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:57.478546 672 raft_consensus.cc:729] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:00:57.478679 672 consensus_queue.cc:260] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:57.478828 672 raft_consensus.cc:3037] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:57.479328 541 raft_consensus.cc:2454] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 2.
I20240502 14:00:57.479442 3284 raft_consensus.cc:2454] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 2.
I20240502 14:00:57.479867 672 raft_consensus.cc:2454] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 478dfa53d7944f38b4e6cabee21b2d01 in term 2.
I20240502 14:00:57.480180 738 leader_election.cc:302] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9, da3f8160557e454f8547f45e80c2f013; no voters:
I20240502 14:00:57.480470 6799 raft_consensus.cc:2781] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:57.480549 6799 raft_consensus.cc:686] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 LEADER]: Becoming Leader. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Running, Role: LEADER
I20240502 14:00:57.480672 6799 consensus_queue.cc:237] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:57.483724 32734 catalog_manager.cc:5484] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: term changed from 1 to 2, leader changed from 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) to 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:57.485131 5098 raft_consensus.cc:1050] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:00:57.485500 5098 consensus_queue.cc:237] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:57.486896 409 raft_consensus.cc:1261] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.487165 3150 raft_consensus.cc:1261] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.487313 6221 consensus_queue.cc:1035] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.487957 6221 consensus_queue.cc:1035] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.488020 804 raft_consensus.cc:1261] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.489434 6221 consensus_queue.cc:1035] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.493132 6065 raft_consensus.cc:2932] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.494011 804 raft_consensus.cc:2932] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.495934 3150 raft_consensus.cc:2932] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.496930 32734 catalog_manager.cc:5484] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.496995 409 raft_consensus.cc:2932] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.501065 672 consensus_queue.cc:237] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:57.502525 405 raft_consensus.cc:1261] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.502975 6221 consensus_queue.cc:1035] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.503593 3149 raft_consensus.cc:1261] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.504385 5711 consensus_queue.cc:1035] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.506844 6221 raft_consensus.cc:2932] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.507295 405 raft_consensus.cc:2932] T 59dd5ee0b3554ac493ce2d4d15a99abd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.509011 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 59dd5ee0b3554ac493ce2d4d15a99abd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.510218 3149 raft_consensus.cc:2932] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.514071 32734 catalog_manager.cc:5484] T 59dd5ee0b3554ac493ce2d4d15a99abd P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 3 to 4, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.517012 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 59dd5ee0b3554ac493ce2d4d15a99abd with delete_type TABLET_DATA_TOMBSTONED (TS 478dfa53d7944f38b4e6cabee21b2d01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39484
I20240502 14:00:57.517220 7084 tablet_replica.cc:330] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:00:57.517383 7084 raft_consensus.cc:2229] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.517503 7084 raft_consensus.cc:2258] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.518186 7084 ts_tablet_manager.cc:1875] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.520064 7084 ts_tablet_manager.cc:1888] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:57.520143 7084 log.cc:1196] T 59dd5ee0b3554ac493ce2d4d15a99abd P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/59dd5ee0b3554ac493ce2d4d15a99abd
I20240502 14:00:57.520663 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 59dd5ee0b3554ac493ce2d4d15a99abd (table rebalancer_extra_table_68 [id=27ea56f6577d467493763287f9354377]) successfully deleted
I20240502 14:00:57.534667 3284 raft_consensus.cc:1203] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:57.539870 541 consensus_queue.cc:237] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
W20240502 14:00:57.541626 477 consensus_peers.cc:485] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2df438b3678241d9a6aa6f0deb2cee2d. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.541671 3284 raft_consensus.cc:1261] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.542248 6834 consensus_queue.cc:1035] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.542577 2886 raft_consensus.cc:1261] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.544057 6834 consensus_queue.cc:1035] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.545221 5655 raft_consensus.cc:2932] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.547139 3284 raft_consensus.cc:2932] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.548987 32729 catalog_manager.cc:5484] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:57.550633 2884 raft_consensus.cc:2932] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.559254 7372 ts_tablet_manager.cc:902] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:57.559801 7372 tablet_copy_client.cc:308] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35251
I20240502 14:00:57.560714 562 tablet_copy_service.cc:140] P c351a89a5fe34e788b9d8064fbffa751: Received BeginTabletCopySession request for tablet db0bf9f6821045e1a259703fc3445226 from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:46139)
I20240502 14:00:57.560807 562 tablet_copy_service.cc:161] P c351a89a5fe34e788b9d8064fbffa751: Beginning new tablet copy session on tablet db0bf9f6821045e1a259703fc3445226 from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:46139: session id = 478dfa53d7944f38b4e6cabee21b2d01-db0bf9f6821045e1a259703fc3445226
I20240502 14:00:57.560931 2886 consensus_queue.cc:237] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:57.561615 562 tablet_copy_source_session.cc:205] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:57.562551 5655 raft_consensus.cc:1050] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:00:57.562716 3019 raft_consensus.cc:1261] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.562754 7372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db0bf9f6821045e1a259703fc3445226. 1 dirs total, 0 dirs full, 0 dirs failed
W20240502 14:00:57.562917 2821 consensus_peers.cc:485] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b4469ea792f4c1f9dce98e47b51f9f4. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.562956 5655 consensus_queue.cc:237] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:57.563542 7206 consensus_queue.cc:1035] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.564515 804 raft_consensus.cc:1261] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.564924 3149 raft_consensus.cc:1261] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.564958 7176 consensus_queue.cc:1035] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.565305 672 raft_consensus.cc:1261] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.565460 7372 tablet_copy_client.cc:783] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:57.565654 7372 tablet_copy_client.cc:647] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:57.565876 803 raft_consensus.cc:1261] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.566190 7176 consensus_queue.cc:1035] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.566870 7181 consensus_queue.cc:1035] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.566875 7206 consensus_queue.cc:1035] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.568702 7372 tablet_copy_client.cc:523] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:57.568713 7181 raft_consensus.cc:2932] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.570098 7372 tablet_bootstrap.cc:492] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:57.570636 7206 raft_consensus.cc:2932] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.570736 672 raft_consensus.cc:2932] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.571539 804 raft_consensus.cc:2932] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.572248 803 raft_consensus.cc:2932] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.575215 3149 raft_consensus.cc:2932] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.575610 32728 catalog_manager.cc:5484] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 4 to 5, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.577698 3017 raft_consensus.cc:2932] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.580919 7372 tablet_bootstrap.cc:492] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:57.581177 7372 tablet_bootstrap.cc:492] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:00:57.581216 541 consensus_queue.cc:237] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:57.581329 7372 ts_tablet_manager.cc:1367] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.001s
I20240502 14:00:57.581849 7372 raft_consensus.cc:348] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:57.582018 7372 raft_consensus.cc:729] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:00:57.582190 7372 consensus_queue.cc:260] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:57.582589 7372 ts_tablet_manager.cc:1398] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 14:00:57.583168 562 tablet_copy_service.cc:342] P c351a89a5fe34e788b9d8064fbffa751: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-db0bf9f6821045e1a259703fc3445226 received from {username='slave'} at 127.26.25.132:46139
I20240502 14:00:57.583240 562 tablet_copy_service.cc:433] P c351a89a5fe34e788b9d8064fbffa751: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-db0bf9f6821045e1a259703fc3445226 on tablet db0bf9f6821045e1a259703fc3445226 with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:00:57.588197 32729 catalog_manager.cc:5484] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.589594 3149 raft_consensus.cc:1261] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:57.590274 803 raft_consensus.cc:1261] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:57.591424 7384 consensus_queue.cc:1035] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.591761 5655 consensus_queue.cc:1035] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.593338 3017 raft_consensus.cc:1203] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:57.597610 5655 raft_consensus.cc:2932] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.599336 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8568aa57fce64ccf89a36deb5b7b0f3b with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.600020 804 raft_consensus.cc:2932] T 8568aa57fce64ccf89a36deb5b7b0f3b P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.601106 32728 catalog_manager.cc:5484] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:57.603878 3149 raft_consensus.cc:2932] T 8568aa57fce64ccf89a36deb5b7b0f3b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.607169 3284 consensus_queue.cc:237] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
W20240502 14:00:57.609020 3219 consensus_peers.cc:485] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9cd00d821b5d4b35ada028a8c83d0ed7. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.609125 3017 raft_consensus.cc:1261] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.609457 2886 raft_consensus.cc:1261] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.609727 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 8568aa57fce64ccf89a36deb5b7b0f3b with delete_type TABLET_DATA_TOMBSTONED (TS 58107b611f8f475a910a6fd54dfd1fb9 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:57.609970 7210 tablet_replica.cc:330] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:57.610146 7210 raft_consensus.cc:2229] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.610271 6814 consensus_queue.cc:1035] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.610339 7210 raft_consensus.cc:2258] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.610592 6282 consensus_queue.cc:1035] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.612550 6282 raft_consensus.cc:2932] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:57.612681 7210 ts_tablet_manager.cc:1875] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.614992 7210 ts_tablet_manager.cc:1888] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:57.616837 7210 log.cc:1196] T 8568aa57fce64ccf89a36deb5b7b0f3b P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/8568aa57fce64ccf89a36deb5b7b0f3b
I20240502 14:00:57.616782 3019 raft_consensus.cc:2932] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:57.617250 32728 catalog_manager.cc:5484] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:57.618018 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 8568aa57fce64ccf89a36deb5b7b0f3b (table rebalance_test_table_9 [id=ff4d6b4c29c045f2b6b5bc172c5fd4d3]) successfully deleted
I20240502 14:00:57.619269 2886 raft_consensus.cc:2932] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:57.635212 7402 ts_tablet_manager.cc:902] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:57.635984 7402 tablet_copy_client.cc:308] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:57.636536 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 618c1b9af9504c03bd250d88c3a2b0ba from peer c351a89a5fe34e788b9d8064fbffa751 ({username='slave'} at 127.26.25.130:50993)
I20240502 14:00:57.636636 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 618c1b9af9504c03bd250d88c3a2b0ba from peer c351a89a5fe34e788b9d8064fbffa751 at {username='slave'} at 127.26.25.130:50993: session id = c351a89a5fe34e788b9d8064fbffa751-618c1b9af9504c03bd250d88c3a2b0ba
I20240502 14:00:57.637601 693 tablet_copy_source_session.cc:205] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:57.638806 7402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 618c1b9af9504c03bd250d88c3a2b0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.641741 7402 tablet_copy_client.cc:783] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:57.642001 7402 tablet_copy_client.cc:647] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:57.643337 7402 tablet_copy_client.cc:523] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:57.644652 7402 tablet_bootstrap.cc:492] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:57.650166 7402 tablet_bootstrap.cc:492] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:57.650493 7402 tablet_bootstrap.cc:492] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751: Bootstrap complete.
I20240502 14:00:57.650662 7402 ts_tablet_manager.cc:1367] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20240502 14:00:57.651209 7402 raft_consensus.cc:348] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:57.651320 7402 raft_consensus.cc:729] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: LEARNER
I20240502 14:00:57.651427 7402 consensus_queue.cc:260] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:57.651741 7402 ts_tablet_manager.cc:1398] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:57.651950 541 consensus_queue.cc:237] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:57.653523 405 raft_consensus.cc:1261] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.654650 672 raft_consensus.cc:1261] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:57.655342 474 consensus_peers.cc:485] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): Couldn't send request to peer 452ecbc085e54db49c3b01c20a96f860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee25d04c50a64ab98282e3659c94bd71. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.655627 6834 consensus_queue.cc:1035] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.655774 7384 consensus_queue.cc:1035] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.659384 6834 raft_consensus.cc:2932] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:57.661661 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session c351a89a5fe34e788b9d8064fbffa751-618c1b9af9504c03bd250d88c3a2b0ba received from {username='slave'} at 127.26.25.130:50993
I20240502 14:00:57.661746 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session c351a89a5fe34e788b9d8064fbffa751-618c1b9af9504c03bd250d88c3a2b0ba on tablet 618c1b9af9504c03bd250d88c3a2b0ba with peer c351a89a5fe34e788b9d8064fbffa751
I20240502 14:00:57.663465 409 raft_consensus.cc:2932] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:57.667129 32729 catalog_manager.cc:5484] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:57.672452 673 raft_consensus.cc:2932] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:57.689841 3150 consensus_queue.cc:237] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
W20240502 14:00:57.691409 3087 consensus_peers.cc:485] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9d974d536ed41bb9cdc0c3abb64a773. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.692034 3019 raft_consensus.cc:1261] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.692214 541 raft_consensus.cc:1261] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.692517 6625 consensus_queue.cc:1035] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.692672 7246 consensus_queue.cc:1035] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.694631 7006 raft_consensus.cc:2932] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.695354 3019 raft_consensus.cc:2932] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.697021 541 raft_consensus.cc:2932] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.701242 32729 catalog_manager.cc:5484] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.713711 3019 consensus_queue.cc:237] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:57.715122 2886 raft_consensus.cc:1261] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:57.715298 2953 consensus_peers.cc:485] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ffed38329ca4e339a849c5ab3b09e8b. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.715544 5282 consensus_queue.cc:1035] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.716066 3150 raft_consensus.cc:1261] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.716526 5282 consensus_queue.cc:1035] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.717917 5282 raft_consensus.cc:2932] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.719780 2886 raft_consensus.cc:2932] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.723340 32728 catalog_manager.cc:5484] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.726751 3150 raft_consensus.cc:2932] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.733516 3150 consensus_queue.cc:237] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:57.735157 5358 raft_consensus.cc:1050] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER c351a89a5fe34e788b9d8064fbffa751 to VOTER
I20240502 14:00:57.735203 2886 raft_consensus.cc:1261] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.735216 405 raft_consensus.cc:1261] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.735617 5358 consensus_queue.cc:237] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:57.735642 6625 consensus_queue.cc:1035] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.735873 7246 consensus_queue.cc:1035] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.737335 3284 raft_consensus.cc:1261] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.737558 541 raft_consensus.cc:1261] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20240502 14:00:57.737787 3084 consensus_peers.cc:485] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a4015371eba240cb8e8f7e2180ac3d27. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.737982 6939 consensus_queue.cc:1035] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.738240 3019 raft_consensus.cc:1261] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:57.738585 7207 consensus_queue.cc:1035] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.738770 6939 consensus_queue.cc:1035] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:57.739585 7006 raft_consensus.cc:2932] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:57.743155 32728 catalog_manager.cc:5484] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:57.743355 405 raft_consensus.cc:2932] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:57.746526 2886 raft_consensus.cc:2932] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:57.750489 7206 raft_consensus.cc:2932] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.752593 3019 raft_consensus.cc:2932] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.755177 541 raft_consensus.cc:2932] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.755510 3281 raft_consensus.cc:2932] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.756584 32729 catalog_manager.cc:5484] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.762739 2886 consensus_queue.cc:237] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:57.764489 3281 raft_consensus.cc:1261] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.764768 541 raft_consensus.cc:1261] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.764875 673 consensus_queue.cc:237] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
W20240502 14:00:57.766235 607 consensus_peers.cc:485] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e673d6e79c0f4c62a393a542473930b1. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.766258 541 raft_consensus.cc:1261] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.766253 6939 consensus_queue.cc:1035] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.766588 6939 consensus_queue.cc:1035] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.766969 5098 consensus_queue.cc:1035] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.767997 7207 raft_consensus.cc:2932] T 184c03808a1e470dba9c6bfa8e04774a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.768389 405 raft_consensus.cc:1261] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:57.768579 3281 raft_consensus.cc:2932] T 184c03808a1e470dba9c6bfa8e04774a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.768821 6065 consensus_queue.cc:1035] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:57.769506 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 184c03808a1e470dba9c6bfa8e04774a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.769853 540 raft_consensus.cc:2932] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.771330 5098 raft_consensus.cc:2932] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:57.771891 405 raft_consensus.cc:2932] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:57.775403 32726 catalog_manager.cc:5484] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:57.777379 32734 catalog_manager.cc:5484] T 184c03808a1e470dba9c6bfa8e04774a P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:57.781095 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 184c03808a1e470dba9c6bfa8e04774a with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:57.781388 7195 tablet_replica.cc:330] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:57.781554 7195 raft_consensus.cc:2229] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.781713 7195 raft_consensus.cc:2258] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.782028 540 raft_consensus.cc:2932] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:57.782322 7195 ts_tablet_manager.cc:1875] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.784564 7195 ts_tablet_manager.cc:1888] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:57.784651 7195 log.cc:1196] T 184c03808a1e470dba9c6bfa8e04774a P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/184c03808a1e470dba9c6bfa8e04774a
I20240502 14:00:57.786170 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 184c03808a1e470dba9c6bfa8e04774a (table rebalancer_extra_table_36 [id=409fb32bfca44ecaa3f9d301ac14ce3a]) successfully deleted
I20240502 14:00:57.791546 405 consensus_queue.cc:237] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:57.793035 3019 raft_consensus.cc:1261] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.793431 3150 raft_consensus.cc:1261] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.793491 5715 consensus_queue.cc:1035] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:57.793879 344 consensus_peers.cc:485] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c1dc8816aa514d1abcce63ce0e0bf357. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.794697 5715 consensus_queue.cc:1035] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.795082 673 consensus_queue.cc:237] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:57.795365 7166 raft_consensus.cc:2932] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.795799 3019 raft_consensus.cc:2932] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.797086 3149 raft_consensus.cc:1261] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:57.797873 540 raft_consensus.cc:1261] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:57.798264 5098 consensus_queue.cc:1035] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.799785 32734 catalog_manager.cc:5484] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:57.801326 6065 raft_consensus.cc:2932] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.802682 5098 consensus_queue.cc:1035] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.803987 3150 raft_consensus.cc:2932] T 7113a6da4229422099dbd604a560fb95 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.804225 540 raft_consensus.cc:2932] T 7113a6da4229422099dbd604a560fb95 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:57.804786 32728 catalog_manager.cc:5484] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 5 to 6, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:57.806332 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7113a6da4229422099dbd604a560fb95 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:57.807907 387 tablet_service.cc:1514] Processing DeleteTablet for tablet 7113a6da4229422099dbd604a560fb95 with delete_type TABLET_DATA_TOMBSTONED (TS f003b732dd0f46958262b42f8d20305e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:57.808120 7259 tablet_replica.cc:330] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:57.808259 7259 raft_consensus.cc:2229] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:57.808418 7259 raft_consensus.cc:2258] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:57.810374 7259 ts_tablet_manager.cc:1875] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:57.812690 7259 ts_tablet_manager.cc:1888] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:57.812755 7259 log.cc:1196] T 7113a6da4229422099dbd604a560fb95 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/7113a6da4229422099dbd604a560fb95
I20240502 14:00:57.813299 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 7113a6da4229422099dbd604a560fb95 (table rebalance_test_table_19 [id=43c818ed117d462dbd829b88fadb4e60]) successfully deleted
I20240502 14:00:57.813843 3150 raft_consensus.cc:2932] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.864375 673 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:00:57.864807 6174 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.937747 3281 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:00:57.938170 6174 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.941660 540 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:00:57.943303 5429 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:57.959647 2886 consensus_queue.cc:237] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:57.961441 3149 raft_consensus.cc:1261] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.961455 673 raft_consensus.cc:1261] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:57.961728 2821 consensus_peers.cc:485] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8cf2d71983204c3c96d112cab1004478. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.961956 7206 consensus_queue.cc:1035] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.962302 7206 consensus_queue.cc:1035] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.963907 7206 raft_consensus.cc:2932] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.964265 673 raft_consensus.cc:2932] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.966325 3149 raft_consensus.cc:2932] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.968312 32729 catalog_manager.cc:5484] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:57.976291 3281 raft_consensus.cc:1203] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:57.980991 673 consensus_queue.cc:237] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:57.982520 406 raft_consensus.cc:1261] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:57.982520 3149 raft_consensus.cc:1261] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:57.982537 609 consensus_peers.cc:485] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a9aac11ac0b44339b6d9b0f84702871. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:57.982903 5098 consensus_queue.cc:1035] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.983049 5711 consensus_queue.cc:1035] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:57.984761 5711 raft_consensus.cc:2932] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.985363 3149 raft_consensus.cc:2932] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.986407 406 raft_consensus.cc:2932] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:57.987638 32728 catalog_manager.cc:5484] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:57.993551 6174 raft_consensus.cc:1050] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01: attempting to promote NON_VOTER 452ecbc085e54db49c3b01c20a96f860 to VOTER
I20240502 14:00:57.993978 6174 consensus_queue.cc:237] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:00:57.995188 3019 raft_consensus.cc:1261] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.995559 406 raft_consensus.cc:1261] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.995813 6799 consensus_queue.cc:1035] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.996284 6799 consensus_queue.cc:1035] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.998731 673 raft_consensus.cc:1261] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:57.999163 6174 consensus_queue.cc:1035] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:57.999876 406 raft_consensus.cc:2932] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:58.000272 673 raft_consensus.cc:2932] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:57.999626 7578 raft_consensus.cc:2932] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:58.004048 32734 catalog_manager.cc:5484] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:58.004819 3149 consensus_queue.cc:237] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:58.005795 3019 raft_consensus.cc:2932] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:58.006669 3281 raft_consensus.cc:1261] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:58.007359 7246 consensus_queue.cc:1035] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.008177 2886 raft_consensus.cc:1261] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:58.008327 3084 consensus_peers.cc:485] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 49595e1db3c44669b438bd0ac1093a62. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:58.010830 7246 raft_consensus.cc:2932] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:58.012485 3281 raft_consensus.cc:2932] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:58.013887 32726 catalog_manager.cc:5484] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:58.016734 6065 raft_consensus.cc:1050] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER da3f8160557e454f8547f45e80c2f013 to VOTER
I20240502 14:00:58.017097 6065 consensus_queue.cc:237] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:58.018410 804 raft_consensus.cc:1261] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:58.018597 540 raft_consensus.cc:1261] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:58.018832 5098 consensus_queue.cc:1035] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:58.019129 5098 consensus_queue.cc:1035] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:58.020381 6625 consensus_queue.cc:1035] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.020565 3281 raft_consensus.cc:1261] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:58.020963 5711 consensus_queue.cc:1035] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:58.021943 2886 raft_consensus.cc:2932] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:58.022087 803 consensus_queue.cc:237] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:00:58.023427 3019 raft_consensus.cc:1261] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:58.024168 5429 consensus_queue.cc:1035] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:58.026127 3019 raft_consensus.cc:2932] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:58.026291 406 raft_consensus.cc:1261] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:58.026785 6174 consensus_queue.cc:1035] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:58.027778 406 raft_consensus.cc:2932] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:58.030156 6221 raft_consensus.cc:2932] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.031575 32726 catalog_manager.cc:5484] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:58.032181 540 raft_consensus.cc:2932] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.032940 3281 raft_consensus.cc:2932] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.034700 32734 catalog_manager.cc:5484] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:58.037345 804 raft_consensus.cc:2932] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.045377 653 tablet_service.cc:1514] Processing DeleteTablet for tablet 85ad896b8838472096851682071c0a9c with delete_type TABLET_DATA_TOMBSTONED (TS 58107b611f8f475a910a6fd54dfd1fb9 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:58.045675 7210 tablet_replica.cc:330] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:58.045827 7210 raft_consensus.cc:2229] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:58.045995 7210 raft_consensus.cc:2258] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:58.046689 7210 ts_tablet_manager.cc:1875] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:58.046787 673 consensus_queue.cc:237] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:58.047947 3281 raft_consensus.cc:1261] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:58.048089 804 raft_consensus.cc:1261] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:58.048403 6221 consensus_queue.cc:1035] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:58.048991 7210 ts_tablet_manager.cc:1888] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:58.049057 7210 log.cc:1196] T 85ad896b8838472096851682071c0a9c P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/85ad896b8838472096851682071c0a9c
I20240502 14:00:58.049388 6221 consensus_queue.cc:1035] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:58.050769 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet 85ad896b8838472096851682071c0a9c (table rebalance_test_table_19 [id=43c818ed117d462dbd829b88fadb4e60]) successfully deleted
I20240502 14:00:58.050899 6221 raft_consensus.cc:2932] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.052266 7578 raft_consensus.cc:2932] T 85ad896b8838472096851682071c0a9c P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:58.053495 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7e9f86a6149d43f3a217769865d9a3d9 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:58.053900 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 85ad896b8838472096851682071c0a9c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:58.054697 804 raft_consensus.cc:2932] T 7e9f86a6149d43f3a217769865d9a3d9 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.055469 32728 catalog_manager.cc:5484] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:58.056982 3283 raft_consensus.cc:2932] T 7e9f86a6149d43f3a217769865d9a3d9 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.061167 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e9f86a6149d43f3a217769865d9a3d9 with delete_type TABLET_DATA_TOMBSTONED (TS c351a89a5fe34e788b9d8064fbffa751 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:58.061923 7594 tablet_replica.cc:330] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:58.062072 7594 raft_consensus.cc:2229] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:58.062232 7594 raft_consensus.cc:2258] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:58.062873 7594 ts_tablet_manager.cc:1875] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:58.066918 7594 ts_tablet_manager.cc:1888] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:58.067051 7594 log.cc:1196] T 7e9f86a6149d43f3a217769865d9a3d9 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/7e9f86a6149d43f3a217769865d9a3d9
I20240502 14:00:58.067641 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 7e9f86a6149d43f3a217769865d9a3d9 (table rebalance_test_table_9 [id=ff4d6b4c29c045f2b6b5bc172c5fd4d3]) successfully deleted
I20240502 14:00:58.070335 7584 ts_tablet_manager.cc:902] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:58.070861 7584 tablet_copy_client.cc:308] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.136:43131
I20240502 14:00:58.071321 3304 tablet_copy_service.cc:140] P da3f8160557e454f8547f45e80c2f013: Received BeginTabletCopySession request for tablet 9cd00d821b5d4b35ada028a8c83d0ed7 from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:56281)
I20240502 14:00:58.071421 3304 tablet_copy_service.cc:161] P da3f8160557e454f8547f45e80c2f013: Beginning new tablet copy session on tablet 9cd00d821b5d4b35ada028a8c83d0ed7 from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:56281: session id = 2b6796cf0e2e496e9dd641beb432f485-9cd00d821b5d4b35ada028a8c83d0ed7
I20240502 14:00:58.072350 3304 tablet_copy_source_session.cc:205] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.073380 7584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd00d821b5d4b35ada028a8c83d0ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.076125 2886 consensus_queue.cc:237] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:58.077183 7584 tablet_copy_client.cc:783] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.077339 7584 tablet_copy_client.cc:647] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.078032 3019 raft_consensus.cc:1261] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:58.078451 6939 consensus_queue.cc:1035] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.078512 3281 raft_consensus.cc:1261] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:58.078840 2819 consensus_peers.cc:485] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d9d248c2c6745489d2656752db9ddd5. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:58.079968 6939 consensus_queue.cc:1035] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.080104 7584 tablet_copy_client.cc:523] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.083284 7584 tablet_bootstrap.cc:492] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:58.087139 5358 raft_consensus.cc:2932] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:58.091845 32728 catalog_manager.cc:5484] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:58.093495 3019 raft_consensus.cc:2932] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:58.095640 7603 ts_tablet_manager.cc:902] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:58.096494 7603 tablet_copy_client.cc:308] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:00:58.096397 3281 raft_consensus.cc:2932] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:58.097313 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 2b4469ea792f4c1f9dce98e47b51f9f4 from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:34853)
I20240502 14:00:58.097406 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 2b4469ea792f4c1f9dce98e47b51f9f4 from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:34853: session id = f003b732dd0f46958262b42f8d20305e-2b4469ea792f4c1f9dce98e47b51f9f4
I20240502 14:00:58.098400 2906 tablet_copy_source_session.cc:205] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.099313 7603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4469ea792f4c1f9dce98e47b51f9f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.102097 7603 tablet_copy_client.cc:783] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.102304 7603 tablet_copy_client.cc:647] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.103786 804 raft_consensus.cc:1203] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:58.105815 7584 tablet_bootstrap.cc:492] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.106119 7584 tablet_bootstrap.cc:492] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:58.106897 7584 ts_tablet_manager.cc:1367] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.000s
I20240502 14:00:58.107542 7584 raft_consensus.cc:348] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:58.107662 7584 raft_consensus.cc:729] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:58.107753 7584 consensus_queue.cc:260] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:58.109989 7584 ts_tablet_manager.cc:1398] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:58.110584 3304 tablet_copy_service.cc:342] P da3f8160557e454f8547f45e80c2f013: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-9cd00d821b5d4b35ada028a8c83d0ed7 received from {username='slave'} at 127.26.25.135:56281
I20240502 14:00:58.110682 3304 tablet_copy_service.cc:433] P da3f8160557e454f8547f45e80c2f013: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-9cd00d821b5d4b35ada028a8c83d0ed7 on tablet 9cd00d821b5d4b35ada028a8c83d0ed7 with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:58.109064 7603 tablet_copy_client.cc:523] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.117005 7603 tablet_bootstrap.cc:492] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:58.120898 3283 consensus_queue.cc:237] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:58.122238 3019 raft_consensus.cc:1261] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:58.122617 3149 raft_consensus.cc:1261] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:58.122885 7615 consensus_queue.cc:1035] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.123306 7615 consensus_queue.cc:1035] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:58.123548 3218 consensus_peers.cc:485] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f2c438f3a854f528ae8555cdb13721b. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:58.124727 7603 tablet_bootstrap.cc:492] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.124966 7603 tablet_bootstrap.cc:492] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:00:58.125144 7603 ts_tablet_manager.cc:1367] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20240502 14:00:58.125268 7396 raft_consensus.cc:2932] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:58.125689 3019 raft_consensus.cc:2932] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:58.125947 3149 raft_consensus.cc:2932] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:00:58.126459 7603 raft_consensus.cc:348] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:58.126621 7603 raft_consensus.cc:729] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:00:58.127905 7603 consensus_queue.cc:260] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:58.129042 7603 ts_tablet_manager.cc:1398] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:58.130059 32729 catalog_manager.cc:5484] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:58.130302 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-2b4469ea792f4c1f9dce98e47b51f9f4 received from {username='slave'} at 127.26.25.129:34853
I20240502 14:00:58.130381 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session f003b732dd0f46958262b42f8d20305e-2b4469ea792f4c1f9dce98e47b51f9f4 on tablet 2b4469ea792f4c1f9dce98e47b51f9f4 with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:00:58.161381 7639 ts_tablet_manager.cc:902] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:58.161931 7644 ts_tablet_manager.cc:902] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:58.162551 7644 tablet_copy_client.cc:308] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35251
I20240502 14:00:58.162781 7639 tablet_copy_client.cc:308] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:58.163030 562 tablet_copy_service.cc:140] P c351a89a5fe34e788b9d8064fbffa751: Received BeginTabletCopySession request for tablet 2df438b3678241d9a6aa6f0deb2cee2d from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:46139)
I20240502 14:00:58.163137 562 tablet_copy_service.cc:161] P c351a89a5fe34e788b9d8064fbffa751: Beginning new tablet copy session on tablet 2df438b3678241d9a6aa6f0deb2cee2d from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:46139: session id = 478dfa53d7944f38b4e6cabee21b2d01-2df438b3678241d9a6aa6f0deb2cee2d
I20240502 14:00:58.163889 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet e673d6e79c0f4c62a393a542473930b1 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:47487)
I20240502 14:00:58.163969 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet e673d6e79c0f4c62a393a542473930b1 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:47487: session id = da3f8160557e454f8547f45e80c2f013-e673d6e79c0f4c62a393a542473930b1
I20240502 14:00:58.164124 562 tablet_copy_source_session.cc:205] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.164839 693 tablet_copy_source_session.cc:205] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.165781 7644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2df438b3678241d9a6aa6f0deb2cee2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.166047 7639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e673d6e79c0f4c62a393a542473930b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.168121 7644 tablet_copy_client.cc:783] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.168262 7644 tablet_copy_client.cc:647] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.168352 7639 tablet_copy_client.cc:783] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.168481 7639 tablet_copy_client.cc:647] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.170763 7639 tablet_copy_client.cc:523] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.171207 7644 tablet_copy_client.cc:523] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.172097 7639 tablet_bootstrap.cc:492] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:58.172521 7644 tablet_bootstrap.cc:492] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:58.177018 7644 tablet_bootstrap.cc:492] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.177254 7644 tablet_bootstrap.cc:492] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:00:58.177409 7644 ts_tablet_manager.cc:1367] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240502 14:00:58.177848 7644 raft_consensus.cc:348] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:58.177983 7644 raft_consensus.cc:729] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:00:58.178084 7644 consensus_queue.cc:260] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:58.178459 7644 ts_tablet_manager.cc:1398] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 14:00:58.178916 562 tablet_copy_service.cc:342] P c351a89a5fe34e788b9d8064fbffa751: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-2df438b3678241d9a6aa6f0deb2cee2d received from {username='slave'} at 127.26.25.132:46139
I20240502 14:00:58.179003 562 tablet_copy_service.cc:433] P c351a89a5fe34e788b9d8064fbffa751: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-2df438b3678241d9a6aa6f0deb2cee2d on tablet 2df438b3678241d9a6aa6f0deb2cee2d with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:00:58.179431 540 raft_consensus.cc:1203] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:58.190234 7644 ts_tablet_manager.cc:902] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:58.190729 7644 tablet_copy_client.cc:308] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:00:58.192122 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet c9d974d536ed41bb9cdc0c3abb64a773 from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:57613)
I20240502 14:00:58.192222 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet c9d974d536ed41bb9cdc0c3abb64a773 from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:57613: session id = 478dfa53d7944f38b4e6cabee21b2d01-c9d974d536ed41bb9cdc0c3abb64a773
I20240502 14:00:58.193228 3172 tablet_copy_source_session.cc:205] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.194643 7644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9d974d536ed41bb9cdc0c3abb64a773. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.196923 7644 tablet_copy_client.cc:783] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.199210 7644 tablet_copy_client.cc:647] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.200349 7644 tablet_copy_client.cc:523] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.201630 7644 tablet_bootstrap.cc:492] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:58.205296 7644 tablet_bootstrap.cc:492] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.205569 7644 tablet_bootstrap.cc:492] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:00:58.205721 7644 ts_tablet_manager.cc:1367] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20240502 14:00:58.206147 7644 raft_consensus.cc:348] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:58.206285 7644 raft_consensus.cc:729] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:00:58.206377 7644 consensus_queue.cc:260] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:58.206650 7644 ts_tablet_manager.cc:1398] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 14:00:58.209065 7639 tablet_bootstrap.cc:492] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.209339 7639 tablet_bootstrap.cc:492] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:00:58.209509 7639 ts_tablet_manager.cc:1367] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.037s user 0.020s sys 0.007s
I20240502 14:00:58.210049 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-c9d974d536ed41bb9cdc0c3abb64a773 received from {username='slave'} at 127.26.25.132:57613
I20240502 14:00:58.210121 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-c9d974d536ed41bb9cdc0c3abb64a773 on tablet c9d974d536ed41bb9cdc0c3abb64a773 with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:00:58.210347 7639 raft_consensus.cc:348] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:58.210517 7639 raft_consensus.cc:729] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:00:58.210657 7639 consensus_queue.cc:260] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:58.211880 7639 ts_tablet_manager.cc:1398] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:58.213732 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-e673d6e79c0f4c62a393a542473930b1 received from {username='slave'} at 127.26.25.136:47487
I20240502 14:00:58.213819 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session da3f8160557e454f8547f45e80c2f013-e673d6e79c0f4c62a393a542473930b1 on tablet e673d6e79c0f4c62a393a542473930b1 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:00:58.269744 7721 ts_tablet_manager.cc:902] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860: Initiating tablet copy from peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:00:58.270331 7721 tablet_copy_client.cc:308] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35251
I20240502 14:00:58.270833 562 tablet_copy_service.cc:140] P c351a89a5fe34e788b9d8064fbffa751: Received BeginTabletCopySession request for tablet ee25d04c50a64ab98282e3659c94bd71 from peer 452ecbc085e54db49c3b01c20a96f860 ({username='slave'} at 127.26.25.134:36853)
I20240502 14:00:58.270946 562 tablet_copy_service.cc:161] P c351a89a5fe34e788b9d8064fbffa751: Beginning new tablet copy session on tablet ee25d04c50a64ab98282e3659c94bd71 from peer 452ecbc085e54db49c3b01c20a96f860 at {username='slave'} at 127.26.25.134:36853: session id = 452ecbc085e54db49c3b01c20a96f860-ee25d04c50a64ab98282e3659c94bd71
I20240502 14:00:58.272154 562 tablet_copy_source_session.cc:205] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.274085 7721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee25d04c50a64ab98282e3659c94bd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.276700 7721 tablet_copy_client.cc:783] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.276913 7721 tablet_copy_client.cc:647] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.278281 7721 tablet_copy_client.cc:523] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.279745 7721 tablet_bootstrap.cc:492] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:58.304787 7721 tablet_bootstrap.cc:492] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.305177 7721 tablet_bootstrap.cc:492] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap complete.
I20240502 14:00:58.305413 7721 ts_tablet_manager.cc:1367] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.026s user 0.026s sys 0.000s
I20240502 14:00:58.305871 7721 raft_consensus.cc:348] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:58.306088 7721 raft_consensus.cc:729] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: LEARNER
I20240502 14:00:58.306218 7721 consensus_queue.cc:260] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:58.306586 7721 ts_tablet_manager.cc:1398] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:58.306727 672 consensus_queue.cc:237] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:58.307992 562 tablet_copy_service.cc:342] P c351a89a5fe34e788b9d8064fbffa751: Request end of tablet copy session 452ecbc085e54db49c3b01c20a96f860-ee25d04c50a64ab98282e3659c94bd71 received from {username='slave'} at 127.26.25.134:36853
I20240502 14:00:58.308082 562 tablet_copy_service.cc:433] P c351a89a5fe34e788b9d8064fbffa751: ending tablet copy session 452ecbc085e54db49c3b01c20a96f860-ee25d04c50a64ab98282e3659c94bd71 on tablet ee25d04c50a64ab98282e3659c94bd71 with peer 452ecbc085e54db49c3b01c20a96f860
I20240502 14:00:58.308370 540 raft_consensus.cc:1261] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:58.308753 5098 consensus_queue.cc:1035] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:58.308897 804 raft_consensus.cc:1261] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:58.309633 5098 consensus_queue.cc:1035] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:00:58.310828 607 consensus_peers.cc:485] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 567df9d664b7423785175fa385f83e53. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:58.312175 6065 raft_consensus.cc:2932] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:58.312592 804 raft_consensus.cc:2932] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:58.315126 540 raft_consensus.cc:2932] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:58.315753 32729 catalog_manager.cc:5484] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:58.316694 7603 ts_tablet_manager.cc:902] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:58.317144 7603 tablet_copy_client.cc:308] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.134:39061
I20240502 14:00:58.317705 3039 tablet_copy_service.cc:140] P 452ecbc085e54db49c3b01c20a96f860: Received BeginTabletCopySession request for tablet 2ffed38329ca4e339a849c5ab3b09e8b from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:35347)
I20240502 14:00:58.317811 3039 tablet_copy_service.cc:161] P 452ecbc085e54db49c3b01c20a96f860: Beginning new tablet copy session on tablet 2ffed38329ca4e339a849c5ab3b09e8b from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:35347: session id = f003b732dd0f46958262b42f8d20305e-2ffed38329ca4e339a849c5ab3b09e8b
I20240502 14:00:58.318682 3039 tablet_copy_source_session.cc:205] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.320783 7603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ffed38329ca4e339a849c5ab3b09e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.322849 7603 tablet_copy_client.cc:783] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.323019 7603 tablet_copy_client.cc:647] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.324148 7603 tablet_copy_client.cc:523] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.325232 7603 tablet_bootstrap.cc:492] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:58.327944 7644 ts_tablet_manager.cc:902] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:00:58.328423 7644 tablet_copy_client.cc:308] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:44715
I20240502 14:00:58.328933 429 tablet_copy_service.cc:140] P f003b732dd0f46958262b42f8d20305e: Received BeginTabletCopySession request for tablet c1dc8816aa514d1abcce63ce0e0bf357 from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:46951)
I20240502 14:00:58.329022 429 tablet_copy_service.cc:161] P f003b732dd0f46958262b42f8d20305e: Beginning new tablet copy session on tablet c1dc8816aa514d1abcce63ce0e0bf357 from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:46951: session id = 478dfa53d7944f38b4e6cabee21b2d01-c1dc8816aa514d1abcce63ce0e0bf357
I20240502 14:00:58.329262 3283 consensus_queue.cc:237] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:58.329962 429 tablet_copy_source_session.cc:205] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.330116 7603 tablet_bootstrap.cc:492] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.330351 7603 tablet_bootstrap.cc:492] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:00:58.330507 7603 ts_tablet_manager.cc:1367] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20240502 14:00:58.330869 7644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1dc8816aa514d1abcce63ce0e0bf357. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.330927 2886 raft_consensus.cc:1261] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:58.331038 7603 raft_consensus.cc:348] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:58.331171 7603 raft_consensus.cc:729] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:00:58.331297 7603 consensus_queue.cc:260] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
W20240502 14:00:58.331321 3216 consensus_peers.cc:485] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 -> Peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): Couldn't send request to peer 452ecbc085e54db49c3b01c20a96f860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f51e005446c64a07b18f0dbf5056afbc. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:58.331621 7603 ts_tablet_manager.cc:1398] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:58.331846 7396 consensus_queue.cc:1035] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.332170 3039 tablet_copy_service.cc:342] P 452ecbc085e54db49c3b01c20a96f860: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-2ffed38329ca4e339a849c5ab3b09e8b received from {username='slave'} at 127.26.25.129:35347
I20240502 14:00:58.332248 3039 tablet_copy_service.cc:433] P 452ecbc085e54db49c3b01c20a96f860: ending tablet copy session f003b732dd0f46958262b42f8d20305e-2ffed38329ca4e339a849c5ab3b09e8b on tablet 2ffed38329ca4e339a849c5ab3b09e8b with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:00:58.333153 7644 tablet_copy_client.cc:783] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.333292 7644 tablet_copy_client.cc:647] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.333565 672 raft_consensus.cc:1261] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:58.334031 7393 consensus_queue.cc:1035] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.334182 7615 raft_consensus.cc:2932] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:58.336345 672 raft_consensus.cc:2932] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:58.337980 32726 catalog_manager.cc:5484] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:58.339725 7644 tablet_copy_client.cc:523] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.340602 7769 ts_tablet_manager.cc:902] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:58.341323 7644 tablet_bootstrap.cc:492] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:58.341696 7769 tablet_copy_client.cc:308] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:00:58.342552 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet a4015371eba240cb8e8f7e2180ac3d27 from peer 58107b611f8f475a910a6fd54dfd1fb9 ({username='slave'} at 127.26.25.131:33211)
I20240502 14:00:58.342656 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet a4015371eba240cb8e8f7e2180ac3d27 from peer 58107b611f8f475a910a6fd54dfd1fb9 at {username='slave'} at 127.26.25.131:33211: session id = 58107b611f8f475a910a6fd54dfd1fb9-a4015371eba240cb8e8f7e2180ac3d27
I20240502 14:00:58.343386 3172 tablet_copy_source_session.cc:205] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.344385 7769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4015371eba240cb8e8f7e2180ac3d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.344214 2886 raft_consensus.cc:2932] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:58.344760 7644 tablet_bootstrap.cc:492] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.344992 7644 tablet_bootstrap.cc:492] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:00:58.345135 7644 ts_tablet_manager.cc:1367] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20240502 14:00:58.345577 7644 raft_consensus.cc:348] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:58.345711 7644 raft_consensus.cc:729] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:00:58.345808 7644 consensus_queue.cc:260] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:58.346534 7769 tablet_copy_client.cc:783] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.346671 7769 tablet_copy_client.cc:647] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.347103 7644 ts_tablet_manager.cc:1398] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:58.347537 429 tablet_copy_service.cc:342] P f003b732dd0f46958262b42f8d20305e: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-c1dc8816aa514d1abcce63ce0e0bf357 received from {username='slave'} at 127.26.25.132:46951
I20240502 14:00:58.347611 429 tablet_copy_service.cc:433] P f003b732dd0f46958262b42f8d20305e: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-c1dc8816aa514d1abcce63ce0e0bf357 on tablet c1dc8816aa514d1abcce63ce0e0bf357 with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:00:58.349349 7769 tablet_copy_client.cc:523] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.350837 7769 tablet_bootstrap.cc:492] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:58.355199 7769 tablet_bootstrap.cc:492] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.355496 7769 tablet_bootstrap.cc:492] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap complete.
I20240502 14:00:58.355649 7769 ts_tablet_manager.cc:1367] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20240502 14:00:58.356490 7769 raft_consensus.cc:348] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:58.356602 7769 raft_consensus.cc:729] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: LEARNER
I20240502 14:00:58.356704 7769 consensus_queue.cc:260] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:58.357009 7769 ts_tablet_manager.cc:1398] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:58.357542 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-a4015371eba240cb8e8f7e2180ac3d27 received from {username='slave'} at 127.26.25.131:33211
I20240502 14:00:58.357625 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-a4015371eba240cb8e8f7e2180ac3d27 on tablet a4015371eba240cb8e8f7e2180ac3d27 with peer 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:00:58.384418 7603 ts_tablet_manager.cc:902] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:58.384886 7603 tablet_copy_client.cc:308] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:00:58.385316 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 8cf2d71983204c3c96d112cab1004478 from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:34853)
I20240502 14:00:58.385419 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 8cf2d71983204c3c96d112cab1004478 from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:34853: session id = f003b732dd0f46958262b42f8d20305e-8cf2d71983204c3c96d112cab1004478
I20240502 14:00:58.386445 2906 tablet_copy_source_session.cc:205] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.387387 7603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cf2d71983204c3c96d112cab1004478. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.389544 7603 tablet_copy_client.cc:783] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.389698 7603 tablet_copy_client.cc:647] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.391388 7603 tablet_copy_client.cc:523] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.392738 7603 tablet_bootstrap.cc:492] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:58.395911 7603 tablet_bootstrap.cc:492] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.396147 7603 tablet_bootstrap.cc:492] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:00:58.396310 7603 ts_tablet_manager.cc:1367] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 14:00:58.396761 7603 raft_consensus.cc:348] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:58.396906 7603 raft_consensus.cc:729] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:00:58.397364 7603 consensus_queue.cc:260] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:58.397725 7603 ts_tablet_manager.cc:1398] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:58.398255 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-8cf2d71983204c3c96d112cab1004478 received from {username='slave'} at 127.26.25.129:34853
I20240502 14:00:58.398339 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session f003b732dd0f46958262b42f8d20305e-8cf2d71983204c3c96d112cab1004478 on tablet 8cf2d71983204c3c96d112cab1004478 with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:00:58.438148 6221 raft_consensus.cc:1050] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER da3f8160557e454f8547f45e80c2f013 to VOTER
I20240502 14:00:58.438519 6221 consensus_queue.cc:237] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:58.439786 3281 raft_consensus.cc:1261] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:58.440083 405 raft_consensus.cc:1261] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:58.440238 5098 consensus_queue.cc:1035] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:58.440943 540 raft_consensus.cc:1261] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:58.441133 5098 consensus_queue.cc:1035] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:58.441766 5098 consensus_queue.cc:1035] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:58.443320 5711 raft_consensus.cc:2932] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.443722 405 raft_consensus.cc:2932] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.444267 541 raft_consensus.cc:2932] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.445557 3281 raft_consensus.cc:2932] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.448468 32732 catalog_manager.cc:5484] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.453768 673 consensus_queue.cc:237] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:58.455124 3281 raft_consensus.cc:1261] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:58.455298 541 raft_consensus.cc:1261] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:58.455472 5711 consensus_queue.cc:1035] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:58.457686 5098 consensus_queue.cc:1035] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:58.457957 6221 raft_consensus.cc:2932] T edcd3af1a04c43d3ab79d539b758ecb0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.459591 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet edcd3af1a04c43d3ab79d539b758ecb0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:58.459604 541 raft_consensus.cc:2932] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.460323 3281 raft_consensus.cc:2932] T edcd3af1a04c43d3ab79d539b758ecb0 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.464107 32728 catalog_manager.cc:5484] T edcd3af1a04c43d3ab79d539b758ecb0 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 5 to 6, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.468187 387 tablet_service.cc:1514] Processing DeleteTablet for tablet edcd3af1a04c43d3ab79d539b758ecb0 with delete_type TABLET_DATA_TOMBSTONED (TS f003b732dd0f46958262b42f8d20305e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43314
I20240502 14:00:58.468694 7860 tablet_replica.cc:330] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:00:58.468850 7860 raft_consensus.cc:2229] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:58.469034 7860 raft_consensus.cc:2258] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:58.469720 7860 ts_tablet_manager.cc:1875] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:58.472046 7860 ts_tablet_manager.cc:1888] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:58.472131 7860 log.cc:1196] T edcd3af1a04c43d3ab79d539b758ecb0 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/edcd3af1a04c43d3ab79d539b758ecb0
I20240502 14:00:58.472651 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet edcd3af1a04c43d3ab79d539b758ecb0 (table rebalance_test_table_3 [id=673d46c73d60440fb2f914f93a51b362]) successfully deleted
I20240502 14:00:58.487624 7866 ts_tablet_manager.cc:902] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:00:58.488171 7866 tablet_copy_client.cc:308] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:00:58.489233 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet 49595e1db3c44669b438bd0ac1093a62 from peer c351a89a5fe34e788b9d8064fbffa751 ({username='slave'} at 127.26.25.130:55453)
I20240502 14:00:58.489341 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet 49595e1db3c44669b438bd0ac1093a62 from peer c351a89a5fe34e788b9d8064fbffa751 at {username='slave'} at 127.26.25.130:55453: session id = c351a89a5fe34e788b9d8064fbffa751-49595e1db3c44669b438bd0ac1093a62
I20240502 14:00:58.490365 3172 tablet_copy_source_session.cc:205] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.496271 7866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49595e1db3c44669b438bd0ac1093a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.498675 7866 tablet_copy_client.cc:783] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.498864 7866 tablet_copy_client.cc:647] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.499755 803 consensus_queue.cc:237] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:58.500111 7866 tablet_copy_client.cc:523] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.501982 7866 tablet_bootstrap.cc:492] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:58.502082 540 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 14:00:58.503019 3283 raft_consensus.cc:1261] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 14:00:58.504282 6174 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:58.506181 7578 consensus_queue.cc:1035] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:58.506405 5429 raft_consensus.cc:2932] T e5bed83e31be4129a19eb8c347514b80 P 478dfa53d7944f38b4e6cabee21b2d01 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.507314 3283 raft_consensus.cc:2932] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.508134 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e5bed83e31be4129a19eb8c347514b80 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:58.509429 540 raft_consensus.cc:2932] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.513438 32729 catalog_manager.cc:5484] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 5 to 7, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New cstate: current_term: 2 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:58.509435 7866 tablet_bootstrap.cc:492] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.515375 7866 tablet_bootstrap.cc:492] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751: Bootstrap complete.
I20240502 14:00:58.515689 7866 ts_tablet_manager.cc:1367] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20240502 14:00:58.516358 653 tablet_service.cc:1514] Processing DeleteTablet for tablet e5bed83e31be4129a19eb8c347514b80 with delete_type TABLET_DATA_TOMBSTONED (TS 58107b611f8f475a910a6fd54dfd1fb9 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:58.516366 7866 raft_consensus.cc:348] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:58.516477 7866 raft_consensus.cc:729] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: LEARNER
I20240502 14:00:58.516602 7210 tablet_replica.cc:330] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:58.516570 7866 consensus_queue.cc:260] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:58.516757 7210 raft_consensus.cc:2229] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:58.516865 7210 raft_consensus.cc:2258] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:58.517264 7866 ts_tablet_manager.cc:1398] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:58.517531 7210 ts_tablet_manager.cc:1875] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:58.519984 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session c351a89a5fe34e788b9d8064fbffa751-49595e1db3c44669b438bd0ac1093a62 received from {username='slave'} at 127.26.25.130:55453
I20240502 14:00:58.520083 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session c351a89a5fe34e788b9d8064fbffa751-49595e1db3c44669b438bd0ac1093a62 on tablet 49595e1db3c44669b438bd0ac1093a62 with peer c351a89a5fe34e788b9d8064fbffa751
I20240502 14:00:58.524606 7210 ts_tablet_manager.cc:1888] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 14:00:58.524686 7210 log.cc:1196] T e5bed83e31be4129a19eb8c347514b80 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/e5bed83e31be4129a19eb8c347514b80
I20240502 14:00:58.525305 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet e5bed83e31be4129a19eb8c347514b80 (table rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]) successfully deleted
I20240502 14:00:58.526172 7769 ts_tablet_manager.cc:902] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:58.526615 7769 tablet_copy_client.cc:308] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:00:58.527035 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 2d9d248c2c6745489d2656752db9ddd5 from peer 58107b611f8f475a910a6fd54dfd1fb9 ({username='slave'} at 127.26.25.131:52409)
I20240502 14:00:58.527127 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 2d9d248c2c6745489d2656752db9ddd5 from peer 58107b611f8f475a910a6fd54dfd1fb9 at {username='slave'} at 127.26.25.131:52409: session id = 58107b611f8f475a910a6fd54dfd1fb9-2d9d248c2c6745489d2656752db9ddd5
I20240502 14:00:58.528174 2906 tablet_copy_source_session.cc:205] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.529044 7769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d9d248c2c6745489d2656752db9ddd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.531054 7769 tablet_copy_client.cc:783] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.531196 7769 tablet_copy_client.cc:647] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.532626 7769 tablet_copy_client.cc:523] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.534092 7769 tablet_bootstrap.cc:492] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:58.537945 7769 tablet_bootstrap.cc:492] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.538180 7769 tablet_bootstrap.cc:492] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap complete.
I20240502 14:00:58.538348 7769 ts_tablet_manager.cc:1367] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 14:00:58.538949 7769 raft_consensus.cc:348] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:58.539085 7769 raft_consensus.cc:729] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: LEARNER
I20240502 14:00:58.539189 7769 consensus_queue.cc:260] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:58.539917 7769 ts_tablet_manager.cc:1398] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 14:00:58.540560 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-2d9d248c2c6745489d2656752db9ddd5 received from {username='slave'} at 127.26.25.131:52409
I20240502 14:00:58.540647 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-2d9d248c2c6745489d2656752db9ddd5 on tablet 2d9d248c2c6745489d2656752db9ddd5 with peer 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:00:58.542709 3283 consensus_queue.cc:237] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:58.544546 2886 raft_consensus.cc:1261] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:58.544564 3216 consensus_peers.cc:485] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b20d05c7433e4f56832233e2fc9adc84. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:58.544903 540 raft_consensus.cc:1261] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:58.545413 7615 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.545600 7766 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.547677 7615 raft_consensus.cc:2932] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:58.549358 540 raft_consensus.cc:2932] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:58.550158 2886 raft_consensus.cc:2932] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:58.550892 406 raft_consensus.cc:1203] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:58.553926 32729 catalog_manager.cc:5484] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:58.557432 7603 ts_tablet_manager.cc:902] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:58.557888 7603 tablet_copy_client.cc:308] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.136:43131
I20240502 14:00:58.559019 3304 tablet_copy_service.cc:140] P da3f8160557e454f8547f45e80c2f013: Received BeginTabletCopySession request for tablet 2f2c438f3a854f528ae8555cdb13721b from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:57497)
I20240502 14:00:58.559121 3304 tablet_copy_service.cc:161] P da3f8160557e454f8547f45e80c2f013: Beginning new tablet copy session on tablet 2f2c438f3a854f528ae8555cdb13721b from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:57497: session id = f003b732dd0f46958262b42f8d20305e-2f2c438f3a854f528ae8555cdb13721b
I20240502 14:00:58.560137 3304 tablet_copy_source_session.cc:205] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.562242 7603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f2c438f3a854f528ae8555cdb13721b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.564280 7603 tablet_copy_client.cc:783] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.564462 7603 tablet_copy_client.cc:647] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.565878 7603 tablet_copy_client.cc:523] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.567766 7603 tablet_bootstrap.cc:492] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:00:58.570086 7644 ts_tablet_manager.cc:902] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:58.570528 7644 tablet_copy_client.cc:308] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:58.571614 7603 tablet_bootstrap.cc:492] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.571897 7603 tablet_bootstrap.cc:492] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:00:58.572073 7603 ts_tablet_manager.cc:1367] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.003s
I20240502 14:00:58.572580 7603 raft_consensus.cc:348] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:58.572705 7603 raft_consensus.cc:729] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:00:58.572810 7603 consensus_queue.cc:260] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:00:58.573081 7603 ts_tablet_manager.cc:1398] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:58.574442 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 9a9aac11ac0b44339b6d9b0f84702871 from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:56893)
I20240502 14:00:58.574568 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 9a9aac11ac0b44339b6d9b0f84702871 from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:56893: session id = 478dfa53d7944f38b4e6cabee21b2d01-9a9aac11ac0b44339b6d9b0f84702871
I20240502 14:00:58.574604 3304 tablet_copy_service.cc:342] P da3f8160557e454f8547f45e80c2f013: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-2f2c438f3a854f528ae8555cdb13721b received from {username='slave'} at 127.26.25.129:57497
I20240502 14:00:58.574658 3304 tablet_copy_service.cc:433] P da3f8160557e454f8547f45e80c2f013: ending tablet copy session f003b732dd0f46958262b42f8d20305e-2f2c438f3a854f528ae8555cdb13721b on tablet 2f2c438f3a854f528ae8555cdb13721b with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:00:58.575532 693 tablet_copy_source_session.cc:205] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.576395 7644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9aac11ac0b44339b6d9b0f84702871. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.578590 7644 tablet_copy_client.cc:783] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.578744 7644 tablet_copy_client.cc:647] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.580387 7644 tablet_copy_client.cc:523] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.581590 7644 tablet_bootstrap.cc:492] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:58.585635 7644 tablet_bootstrap.cc:492] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.585902 7644 tablet_bootstrap.cc:492] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:00:58.586066 7644 ts_tablet_manager.cc:1367] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20240502 14:00:58.586609 7644 raft_consensus.cc:348] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:58.586848 7644 raft_consensus.cc:729] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:00:58.586989 7644 consensus_queue.cc:260] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:58.587352 7644 ts_tablet_manager.cc:1398] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 14:00:58.588181 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-9a9aac11ac0b44339b6d9b0f84702871 received from {username='slave'} at 127.26.25.132:56893
I20240502 14:00:58.588256 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-9a9aac11ac0b44339b6d9b0f84702871 on tablet 9a9aac11ac0b44339b6d9b0f84702871 with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:00:58.600617 3150 raft_consensus.cc:1203] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:58.615324 7384 raft_consensus.cc:1050] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:00:58.615780 7384 consensus_queue.cc:237] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:58.617233 803 raft_consensus.cc:1261] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:58.617456 2886 raft_consensus.cc:1261] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:58.617627 7181 consensus_queue.cc:1035] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:58.617848 3150 raft_consensus.cc:1261] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:58.618326 7181 consensus_queue.cc:1035] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:58.618667 7181 consensus_queue.cc:1035] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:58.627000 7384 raft_consensus.cc:2932] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:58.629209 803 raft_consensus.cc:2932] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:58.630553 2886 raft_consensus.cc:2932] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:58.631151 32728 catalog_manager.cc:5484] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:58.636822 3150 raft_consensus.cc:2932] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:58.640679 540 consensus_queue.cc:237] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:58.641935 3150 raft_consensus.cc:1261] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:58.642434 803 raft_consensus.cc:1261] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:58.643080 7957 consensus_queue.cc:1035] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:58.643371 7952 consensus_queue.cc:1035] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:58.645071 7952 raft_consensus.cc:2932] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:58.646811 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet db0bf9f6821045e1a259703fc3445226 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:58.647512 3150 raft_consensus.cc:2932] T db0bf9f6821045e1a259703fc3445226 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:58.648119 804 raft_consensus.cc:2932] T db0bf9f6821045e1a259703fc3445226 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:58.648972 32726 catalog_manager.cc:5484] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:58.655385 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet db0bf9f6821045e1a259703fc3445226 with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:58.659583 7963 tablet_replica.cc:330] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:58.659759 7963 raft_consensus.cc:2229] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:58.663554 7963 raft_consensus.cc:2258] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:58.664258 7963 ts_tablet_manager.cc:1875] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:58.664753 804 raft_consensus.cc:1203] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:58.667502 7963 ts_tablet_manager.cc:1888] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:58.667625 7963 log.cc:1196] T db0bf9f6821045e1a259703fc3445226 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/db0bf9f6821045e1a259703fc3445226
I20240502 14:00:58.668227 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet db0bf9f6821045e1a259703fc3445226 (table rebalancer_extra_table_22 [id=de113df849174bb5a42c2527f02cac0d]) successfully deleted
I20240502 14:00:58.686070 6065 raft_consensus.cc:1050] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER c351a89a5fe34e788b9d8064fbffa751 to VOTER
I20240502 14:00:58.686527 6065 consensus_queue.cc:237] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:58.687956 2886 raft_consensus.cc:1261] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:58.687958 3283 raft_consensus.cc:1261] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:58.688380 5098 consensus_queue.cc:1035] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:58.688539 540 raft_consensus.cc:1261] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:58.689373 5098 consensus_queue.cc:1035] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:58.691188 6065 consensus_queue.cc:1035] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:58.694303 5098 raft_consensus.cc:2932] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:58.696878 3283 raft_consensus.cc:2932] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:58.698439 2884 raft_consensus.cc:2932] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:58.698561 540 raft_consensus.cc:2932] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:58.698679 32726 catalog_manager.cc:5484] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:58.704777 673 consensus_queue.cc:237] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:58.706336 2884 raft_consensus.cc:1261] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:58.706383 3019 raft_consensus.cc:1203] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:58.706797 6065 consensus_queue.cc:1035] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:58.706923 540 raft_consensus.cc:1261] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:58.708846 7982 consensus_queue.cc:1035] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:58.710950 6065 raft_consensus.cc:2932] T 618c1b9af9504c03bd250d88c3a2b0ba P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:58.712361 2886 raft_consensus.cc:2932] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:58.712839 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 618c1b9af9504c03bd250d88c3a2b0ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:58.717058 32729 catalog_manager.cc:5484] T 618c1b9af9504c03bd250d88c3a2b0ba P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:58.719687 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet 618c1b9af9504c03bd250d88c3a2b0ba with delete_type TABLET_DATA_TOMBSTONED (TS da3f8160557e454f8547f45e80c2f013 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47052
I20240502 14:00:58.719774 540 raft_consensus.cc:2932] T 618c1b9af9504c03bd250d88c3a2b0ba P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:58.720160 7984 tablet_replica.cc:330] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:00:58.720499 7984 raft_consensus.cc:2229] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:58.720729 7984 raft_consensus.cc:2258] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:58.721370 3283 raft_consensus.cc:1203] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:58.723441 7984 ts_tablet_manager.cc:1875] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:58.727902 7984 ts_tablet_manager.cc:1888] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:58.728062 7984 log.cc:1196] T 618c1b9af9504c03bd250d88c3a2b0ba P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/618c1b9af9504c03bd250d88c3a2b0ba
I20240502 14:00:58.728717 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 618c1b9af9504c03bd250d88c3a2b0ba (table rebalancer_extra_table_34 [id=2b22704f8c7e447d9b7e1edd7429f8e6]) successfully deleted
I20240502 14:00:58.735966 409 raft_consensus.cc:1203] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:58.780751 803 raft_consensus.cc:1203] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:58.784210 8031 ts_tablet_manager.cc:902] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:58.784933 8031 tablet_copy_client.cc:308] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:58.785835 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 567df9d664b7423785175fa385f83e53 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:47487)
I20240502 14:00:58.785974 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 567df9d664b7423785175fa385f83e53 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:47487: session id = da3f8160557e454f8547f45e80c2f013-567df9d664b7423785175fa385f83e53
I20240502 14:00:58.786880 693 tablet_copy_source_session.cc:205] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.787860 8031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567df9d664b7423785175fa385f83e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.790262 8031 tablet_copy_client.cc:783] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.790437 2886 consensus_queue.cc:237] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:58.790578 8031 tablet_copy_client.cc:647] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
W20240502 14:00:58.792351 2822 consensus_peers.cc:485] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5e922f7cee8046e7b4263dd57b8d1185. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:58.792398 3283 raft_consensus.cc:1261] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:58.792481 8031 tablet_copy_client.cc:523] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.792532 3019 raft_consensus.cc:1261] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:58.792963 6939 consensus_queue.cc:1035] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.793167 7207 consensus_queue.cc:1035] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.794710 8031 tablet_bootstrap.cc:492] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:00:58.798436 6939 raft_consensus.cc:2932] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:58.798923 3019 raft_consensus.cc:2932] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:58.802712 32732 catalog_manager.cc:5484] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:58.806910 7721 ts_tablet_manager.cc:902] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860: Initiating tablet copy from peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:58.807368 7721 tablet_copy_client.cc:308] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.136:43131
I20240502 14:00:58.807971 3304 tablet_copy_service.cc:140] P da3f8160557e454f8547f45e80c2f013: Received BeginTabletCopySession request for tablet f51e005446c64a07b18f0dbf5056afbc from peer 452ecbc085e54db49c3b01c20a96f860 ({username='slave'} at 127.26.25.134:42185)
I20240502 14:00:58.808091 3304 tablet_copy_service.cc:161] P da3f8160557e454f8547f45e80c2f013: Beginning new tablet copy session on tablet f51e005446c64a07b18f0dbf5056afbc from peer 452ecbc085e54db49c3b01c20a96f860 at {username='slave'} at 127.26.25.134:42185: session id = 452ecbc085e54db49c3b01c20a96f860-f51e005446c64a07b18f0dbf5056afbc
I20240502 14:00:58.808315 3283 raft_consensus.cc:2932] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:58.809000 3304 tablet_copy_source_session.cc:205] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.810206 7721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f51e005446c64a07b18f0dbf5056afbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.812430 409 raft_consensus.cc:1203] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:58.812636 7721 tablet_copy_client.cc:783] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.812783 7721 tablet_copy_client.cc:647] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.821256 7721 tablet_copy_client.cc:523] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.824167 7721 tablet_bootstrap.cc:492] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:00:58.829694 3019 consensus_queue.cc:237] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:58.831298 409 raft_consensus.cc:1261] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:58.831503 2951 consensus_peers.cc:485] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 638cb0f08414491ea80139eb51406dfe. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:58.831749 803 raft_consensus.cc:1261] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:58.831779 7995 consensus_queue.cc:1035] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.832180 7995 consensus_queue.cc:1035] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:58.832814 8031 tablet_bootstrap.cc:492] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.833215 8031 tablet_bootstrap.cc:492] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:00:58.833477 8031 ts_tablet_manager.cc:1367] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.039s user 0.024s sys 0.000s
I20240502 14:00:58.833634 7721 tablet_bootstrap.cc:492] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.834189 8031 raft_consensus.cc:348] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:58.834343 8031 raft_consensus.cc:729] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:00:58.834383 804 raft_consensus.cc:1203] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:58.834450 8031 consensus_queue.cc:260] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:58.834877 7721 tablet_bootstrap.cc:492] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860: Bootstrap complete.
I20240502 14:00:58.835070 7721 ts_tablet_manager.cc:1367] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20240502 14:00:58.835518 7721 raft_consensus.cc:348] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:58.835686 7721 raft_consensus.cc:729] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: LEARNER
I20240502 14:00:58.835844 7721 consensus_queue.cc:260] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:58.837440 7721 ts_tablet_manager.cc:1398] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:58.838299 3304 tablet_copy_service.cc:342] P da3f8160557e454f8547f45e80c2f013: Request end of tablet copy session 452ecbc085e54db49c3b01c20a96f860-f51e005446c64a07b18f0dbf5056afbc received from {username='slave'} at 127.26.25.134:42185
I20240502 14:00:58.839460 3304 tablet_copy_service.cc:433] P da3f8160557e454f8547f45e80c2f013: ending tablet copy session 452ecbc085e54db49c3b01c20a96f860-f51e005446c64a07b18f0dbf5056afbc on tablet f51e005446c64a07b18f0dbf5056afbc with peer 452ecbc085e54db49c3b01c20a96f860
I20240502 14:00:58.840201 8031 ts_tablet_manager.cc:1398] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:00:58.842603 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-567df9d664b7423785175fa385f83e53 received from {username='slave'} at 127.26.25.136:47487
I20240502 14:00:58.842685 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session da3f8160557e454f8547f45e80c2f013-567df9d664b7423785175fa385f83e53 on tablet 567df9d664b7423785175fa385f83e53 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:00:58.843467 5282 raft_consensus.cc:2932] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:58.844000 803 raft_consensus.cc:2932] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:58.846187 409 raft_consensus.cc:2932] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:58.848614 32728 catalog_manager.cc:5484] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:58.885654 541 raft_consensus.cc:1203] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:58.953958 673 raft_consensus.cc:1203] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:58.989199 7769 ts_tablet_manager.cc:902] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9: Initiating tablet copy from peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:00:58.989706 7769 tablet_copy_client.cc:308] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.136:43131
I20240502 14:00:58.990216 3304 tablet_copy_service.cc:140] P da3f8160557e454f8547f45e80c2f013: Received BeginTabletCopySession request for tablet b20d05c7433e4f56832233e2fc9adc84 from peer 58107b611f8f475a910a6fd54dfd1fb9 ({username='slave'} at 127.26.25.131:54755)
I20240502 14:00:58.990332 3304 tablet_copy_service.cc:161] P da3f8160557e454f8547f45e80c2f013: Beginning new tablet copy session on tablet b20d05c7433e4f56832233e2fc9adc84 from peer 58107b611f8f475a910a6fd54dfd1fb9 at {username='slave'} at 127.26.25.131:54755: session id = 58107b611f8f475a910a6fd54dfd1fb9-b20d05c7433e4f56832233e2fc9adc84
I20240502 14:00:58.991068 3304 tablet_copy_source_session.cc:205] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:58.991888 7769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b20d05c7433e4f56832233e2fc9adc84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:58.993551 7769 tablet_copy_client.cc:783] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:58.993708 7769 tablet_copy_client.cc:647] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:58.994824 7769 tablet_copy_client.cc:523] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:58.995787 7769 tablet_bootstrap.cc:492] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:00:58.998947 7769 tablet_bootstrap.cc:492] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:58.999171 7769 tablet_bootstrap.cc:492] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap complete.
I20240502 14:00:58.999320 7769 ts_tablet_manager.cc:1367] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20240502 14:00:58.999686 7769 raft_consensus.cc:348] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:58.999795 7769 raft_consensus.cc:729] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: LEARNER
I20240502 14:00:58.999920 7769 consensus_queue.cc:260] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:59.000185 7769 ts_tablet_manager.cc:1398] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:59.000741 3304 tablet_copy_service.cc:342] P da3f8160557e454f8547f45e80c2f013: Request end of tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-b20d05c7433e4f56832233e2fc9adc84 received from {username='slave'} at 127.26.25.131:54755
I20240502 14:00:59.000830 3304 tablet_copy_service.cc:433] P da3f8160557e454f8547f45e80c2f013: ending tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-b20d05c7433e4f56832233e2fc9adc84 on tablet b20d05c7433e4f56832233e2fc9adc84 with peer 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:00:59.048585 6625 raft_consensus.cc:1050] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:00:59.049059 6625 consensus_queue.cc:237] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:59.050742 803 raft_consensus.cc:1261] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.051357 7006 consensus_queue.cc:1035] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.051414 541 raft_consensus.cc:1261] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.051473 673 raft_consensus.cc:1203] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:59.051811 7246 consensus_queue.cc:1035] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.052024 3019 raft_consensus.cc:1261] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.053035 7246 consensus_queue.cc:1035] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.060473 7246 raft_consensus.cc:2932] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.060986 3019 raft_consensus.cc:2932] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.062963 803 raft_consensus.cc:2932] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.063078 541 raft_consensus.cc:2932] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.065258 32728 catalog_manager.cc:5484] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.070310 3150 consensus_queue.cc:237] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:59.071655 803 raft_consensus.cc:1261] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.071750 3017 raft_consensus.cc:1261] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.072396 6625 consensus_queue.cc:1035] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.072777 6625 consensus_queue.cc:1035] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.074699 6625 raft_consensus.cc:2932] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.076618 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c9d974d536ed41bb9cdc0c3abb64a773 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.077610 3019 raft_consensus.cc:2932] T c9d974d536ed41bb9cdc0c3abb64a773 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.078111 803 raft_consensus.cc:2932] T c9d974d536ed41bb9cdc0c3abb64a773 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.078382 32729 catalog_manager.cc:5484] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.082549 520 tablet_service.cc:1514] Processing DeleteTablet for tablet c9d974d536ed41bb9cdc0c3abb64a773 with delete_type TABLET_DATA_TOMBSTONED (TS c351a89a5fe34e788b9d8064fbffa751 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:59.088073 8160 tablet_replica.cc:330] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:59.088238 8160 raft_consensus.cc:2229] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.088388 8160 raft_consensus.cc:2258] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.088963 8160 ts_tablet_manager.cc:1875] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.091663 8160 ts_tablet_manager.cc:1888] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.091742 8160 log.cc:1196] T c9d974d536ed41bb9cdc0c3abb64a773 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/c9d974d536ed41bb9cdc0c3abb64a773
I20240502 14:00:59.092288 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet c9d974d536ed41bb9cdc0c3abb64a773 (table rebalancer_extra_table_52 [id=d621bb0c4e764e628cf845d1ed77f115]) successfully deleted
I20240502 14:00:59.118327 7957 raft_consensus.cc:1050] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751: attempting to promote NON_VOTER 452ecbc085e54db49c3b01c20a96f860 to VOTER
I20240502 14:00:59.118753 7957 consensus_queue.cc:237] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:00:59.120378 409 raft_consensus.cc:1261] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:59.120407 3019 raft_consensus.cc:1261] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:59.120419 673 raft_consensus.cc:1261] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:59.120851 7953 consensus_queue.cc:1035] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:59.121043 7181 consensus_queue.cc:1035] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:59.121206 7952 consensus_queue.cc:1035] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:59.123495 7953 raft_consensus.cc:2932] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.124544 409 raft_consensus.cc:2932] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.125211 3019 raft_consensus.cc:2932] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.127530 32729 catalog_manager.cc:5484] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:59.130879 541 consensus_queue.cc:237] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:00:59.132407 803 raft_consensus.cc:1203] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:59.132863 6939 raft_consensus.cc:1050] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:00:59.132910 405 raft_consensus.cc:1261] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:59.133288 6939 consensus_queue.cc:237] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:59.134292 7957 consensus_queue.cc:1035] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:59.134903 804 raft_consensus.cc:1261] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.135277 3017 raft_consensus.cc:1261] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.135668 7207 consensus_queue.cc:1035] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.135732 673 raft_consensus.cc:2932] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.137012 7206 consensus_queue.cc:1035] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.137986 406 raft_consensus.cc:1261] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.138672 7207 consensus_queue.cc:1035] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.139142 3019 raft_consensus.cc:1261] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:59.140668 7957 consensus_queue.cc:1035] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:59.140888 7206 raft_consensus.cc:2932] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.142467 7952 raft_consensus.cc:2932] T ee25d04c50a64ab98282e3659c94bd71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.142923 404 raft_consensus.cc:2932] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.144511 3019 raft_consensus.cc:2932] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.144670 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ee25d04c50a64ab98282e3659c94bd71 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.144511 3017 raft_consensus.cc:2932] T ee25d04c50a64ab98282e3659c94bd71 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.145134 405 raft_consensus.cc:2932] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.147729 32726 catalog_manager.cc:5484] T ee25d04c50a64ab98282e3659c94bd71 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 5 to 6, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.148128 803 raft_consensus.cc:2932] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.151707 653 tablet_service.cc:1514] Processing DeleteTablet for tablet ee25d04c50a64ab98282e3659c94bd71 with delete_type TABLET_DATA_TOMBSTONED (TS 58107b611f8f475a910a6fd54dfd1fb9 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:59.151595 32728 catalog_manager.cc:5484] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.157837 8171 tablet_replica.cc:330] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:59.158016 8171 raft_consensus.cc:2229] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.158246 8171 raft_consensus.cc:2258] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.159194 8171 ts_tablet_manager.cc:1875] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.160642 2884 consensus_queue.cc:237] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:59.161513 8171 ts_tablet_manager.cc:1888] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:59.161619 8171 log.cc:1196] T ee25d04c50a64ab98282e3659c94bd71 P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/ee25d04c50a64ab98282e3659c94bd71
I20240502 14:00:59.161787 405 raft_consensus.cc:1261] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.162292 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet ee25d04c50a64ab98282e3659c94bd71 (table rebalance_test_table_7 [id=3dec3eec8fda4142913bf2d106d579b1]) successfully deleted
I20240502 14:00:59.162385 803 raft_consensus.cc:1261] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.162664 7206 consensus_queue.cc:1035] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.163098 8167 consensus_queue.cc:1035] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.164716 6939 raft_consensus.cc:2932] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.165190 405 raft_consensus.cc:2932] T 2b4469ea792f4c1f9dce98e47b51f9f4 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.165171 803 raft_consensus.cc:2932] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.166420 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2b4469ea792f4c1f9dce98e47b51f9f4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.168500 32731 catalog_manager.cc:5484] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:59.171113 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b4469ea792f4c1f9dce98e47b51f9f4 with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:59.171491 8172 tablet_replica.cc:330] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:59.171641 8172 raft_consensus.cc:2229] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.171784 8172 raft_consensus.cc:2258] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.172578 8172 ts_tablet_manager.cc:1875] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.174479 8172 ts_tablet_manager.cc:1888] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.174544 8172 log.cc:1196] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/2b4469ea792f4c1f9dce98e47b51f9f4
I20240502 14:00:59.175055 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 2b4469ea792f4c1f9dce98e47b51f9f4 (table rebalancer_extra_table_16 [id=0b8348df8f78415b994322df0d6d1090]) successfully deleted
I20240502 14:00:59.175323 405 raft_consensus.cc:1203] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:59.213896 7953 raft_consensus.cc:1050] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:00:59.214354 7953 consensus_queue.cc:237] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:59.215225 7396 raft_consensus.cc:1050] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:00:59.215627 7396 consensus_queue.cc:237] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:59.215880 3283 raft_consensus.cc:1261] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.216246 7952 consensus_queue.cc:1035] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.217159 3017 raft_consensus.cc:1261] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.217550 2886 raft_consensus.cc:1261] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.217789 2884 raft_consensus.cc:1261] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.217759 7615 consensus_queue.cc:1035] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.218559 803 raft_consensus.cc:1261] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.218860 7615 consensus_queue.cc:1035] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.219358 3150 raft_consensus.cc:1261] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.220497 7396 consensus_queue.cc:1035] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.220520 7957 consensus_queue.cc:1035] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.220840 7952 consensus_queue.cc:1035] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.220890 7396 raft_consensus.cc:2932] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:59.221448 2884 raft_consensus.cc:2932] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:59.223783 3017 raft_consensus.cc:2932] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:59.225618 32729 catalog_manager.cc:5484] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.228869 7952 raft_consensus.cc:2932] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.229166 803 raft_consensus.cc:2932] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.232236 3283 raft_consensus.cc:2932] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.232308 32732 catalog_manager.cc:5484] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:59.235150 2884 raft_consensus.cc:2932] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.237370 541 consensus_queue.cc:237] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:59.238791 3283 raft_consensus.cc:1203] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:59.240197 3150 raft_consensus.cc:2932] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:59.242475 803 raft_consensus.cc:1261] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.242856 7957 consensus_queue.cc:1035] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.244812 7957 raft_consensus.cc:2932] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.245154 3281 raft_consensus.cc:1261] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.246804 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2df438b3678241d9a6aa6f0deb2cee2d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.248678 3283 consensus_queue.cc:237] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:00:59.248733 32726 catalog_manager.cc:5484] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:59.249626 803 raft_consensus.cc:2932] T 2df438b3678241d9a6aa6f0deb2cee2d P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.250470 3017 raft_consensus.cc:1261] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.251550 7396 consensus_queue.cc:1035] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.251646 7384 consensus_queue.cc:1035] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.252409 3150 raft_consensus.cc:1261] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.254592 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 2df438b3678241d9a6aa6f0deb2cee2d with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:59.254601 3283 raft_consensus.cc:2932] T 2df438b3678241d9a6aa6f0deb2cee2d P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.256044 7396 consensus_queue.cc:1035] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.256510 7396 raft_consensus.cc:2932] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:59.258090 32719 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9cd00d821b5d4b35ada028a8c83d0ed7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.258059 3150 raft_consensus.cc:2932] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:59.260129 3017 raft_consensus.cc:2932] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:00:59.262271 8222 tablet_replica.cc:330] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:59.262440 8222 raft_consensus.cc:2229] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.262585 8222 raft_consensus.cc:2258] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.263613 32731 catalog_manager.cc:5484] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:59.266835 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cd00d821b5d4b35ada028a8c83d0ed7 with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:59.267164 8222 ts_tablet_manager.cc:1875] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.269419 8222 ts_tablet_manager.cc:1888] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.269541 8222 log.cc:1196] T 2df438b3678241d9a6aa6f0deb2cee2d P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/2df438b3678241d9a6aa6f0deb2cee2d
I20240502 14:00:59.270035 8222 tablet_replica.cc:330] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:59.270105 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 2df438b3678241d9a6aa6f0deb2cee2d (table rebalancer_extra_table_42 [id=a6ece96546b04c28ab94cb79046f2008]) successfully deleted
I20240502 14:00:59.270215 8222 raft_consensus.cc:2229] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.270362 8222 raft_consensus.cc:2258] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.270912 8222 ts_tablet_manager.cc:1875] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.272845 8222 ts_tablet_manager.cc:1888] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.272912 8222 log.cc:1196] T 9cd00d821b5d4b35ada028a8c83d0ed7 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/9cd00d821b5d4b35ada028a8c83d0ed7
I20240502 14:00:59.273489 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 9cd00d821b5d4b35ada028a8c83d0ed7 (table rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]) successfully deleted
I20240502 14:00:59.295023 6065 raft_consensus.cc:1050] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER da3f8160557e454f8547f45e80c2f013 to VOTER
I20240502 14:00:59.295495 6065 consensus_queue.cc:237] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:59.297298 405 raft_consensus.cc:1261] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:59.297582 541 raft_consensus.cc:1261] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:59.297902 5098 consensus_queue.cc:1035] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:59.299104 3283 raft_consensus.cc:1261] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:59.298146 6065 consensus_queue.cc:1035] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:59.302258 7982 consensus_queue.cc:1035] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:59.308045 6065 raft_consensus.cc:2932] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.308526 540 raft_consensus.cc:2932] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.310956 404 raft_consensus.cc:2932] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.312461 32729 catalog_manager.cc:5484] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.312623 3283 raft_consensus.cc:2932] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.318866 8259 ts_tablet_manager.cc:902] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751: Initiating tablet copy from peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:00:59.319386 8259 tablet_copy_client.cc:308] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.134:39061
I20240502 14:00:59.319876 3039 tablet_copy_service.cc:140] P 452ecbc085e54db49c3b01c20a96f860: Received BeginTabletCopySession request for tablet 638cb0f08414491ea80139eb51406dfe from peer c351a89a5fe34e788b9d8064fbffa751 ({username='slave'} at 127.26.25.130:33995)
I20240502 14:00:59.319989 3039 tablet_copy_service.cc:161] P 452ecbc085e54db49c3b01c20a96f860: Beginning new tablet copy session on tablet 638cb0f08414491ea80139eb51406dfe from peer c351a89a5fe34e788b9d8064fbffa751 at {username='slave'} at 127.26.25.130:33995: session id = c351a89a5fe34e788b9d8064fbffa751-638cb0f08414491ea80139eb51406dfe
I20240502 14:00:59.320417 673 consensus_queue.cc:237] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:59.320950 3039 tablet_copy_source_session.cc:205] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:59.321854 3283 raft_consensus.cc:1261] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:59.322242 7982 consensus_queue.cc:1035] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:59.322503 404 raft_consensus.cc:1261] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:59.323429 8259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638cb0f08414491ea80139eb51406dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:59.323992 5098 raft_consensus.cc:2932] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.324354 3283 raft_consensus.cc:2932] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.326340 6065 consensus_queue.cc:1035] T e673d6e79c0f4c62a393a542473930b1 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:59.326385 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e673d6e79c0f4c62a393a542473930b1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.326745 8259 tablet_copy_client.cc:783] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:59.327368 405 raft_consensus.cc:2932] T e673d6e79c0f4c62a393a542473930b1 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.328456 8259 tablet_copy_client.cc:647] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:59.328873 32731 catalog_manager.cc:5484] T e673d6e79c0f4c62a393a542473930b1 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.329991 8167 raft_consensus.cc:1050] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:00:59.330438 8167 consensus_queue.cc:237] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:59.331796 520 tablet_service.cc:1514] Processing DeleteTablet for tablet e673d6e79c0f4c62a393a542473930b1 with delete_type TABLET_DATA_TOMBSTONED (TS c351a89a5fe34e788b9d8064fbffa751 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:59.332212 8160 tablet_replica.cc:330] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:59.332258 673 raft_consensus.cc:1261] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.332340 8160 raft_consensus.cc:2229] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.332501 8160 raft_consensus.cc:2258] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.332530 3150 raft_consensus.cc:1261] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.332751 6939 consensus_queue.cc:1035] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.333478 6939 consensus_queue.cc:1035] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.336055 8259 tablet_copy_client.cc:523] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:59.336110 8160 ts_tablet_manager.cc:1875] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.337366 8259 tablet_bootstrap.cc:492] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:00:59.338096 8160 ts_tablet_manager.cc:1888] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:59.338156 8160 log.cc:1196] T e673d6e79c0f4c62a393a542473930b1 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/e673d6e79c0f4c62a393a542473930b1
I20240502 14:00:59.338790 7206 raft_consensus.cc:2932] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.339306 673 raft_consensus.cc:2932] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.340659 3150 raft_consensus.cc:2932] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.342298 405 raft_consensus.cc:1261] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.342693 7207 consensus_queue.cc:1035] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.343298 32729 catalog_manager.cc:5484] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.343772 405 raft_consensus.cc:2932] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.346218 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet e673d6e79c0f4c62a393a542473930b1 (table rebalance_test_table_1 [id=2b9baba891764826815587748cbcfcf1]) successfully deleted
I20240502 14:00:59.347087 7995 raft_consensus.cc:1050] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:00:59.347523 7995 consensus_queue.cc:237] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:59.349031 3150 raft_consensus.cc:1261] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.349198 2884 raft_consensus.cc:1261] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.349388 404 raft_consensus.cc:1261] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.349673 8275 consensus_queue.cc:1035] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.349826 5282 consensus_queue.cc:1035] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.350423 5282 consensus_queue.cc:1035] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.352984 8259 tablet_bootstrap.cc:492] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:59.353274 8259 tablet_bootstrap.cc:492] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751: Bootstrap complete.
I20240502 14:00:59.353463 8259 ts_tablet_manager.cc:1367] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20240502 14:00:59.354351 8259 raft_consensus.cc:348] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:59.354477 8259 raft_consensus.cc:729] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: LEARNER
I20240502 14:00:59.354583 8259 consensus_queue.cc:260] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:59.354781 5282 raft_consensus.cc:2932] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.355015 8259 ts_tablet_manager.cc:1398] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:59.356178 2884 raft_consensus.cc:2932] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.356343 3150 raft_consensus.cc:2932] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.356848 405 raft_consensus.cc:2932] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.357560 3039 tablet_copy_service.cc:342] P 452ecbc085e54db49c3b01c20a96f860: Request end of tablet copy session c351a89a5fe34e788b9d8064fbffa751-638cb0f08414491ea80139eb51406dfe received from {username='slave'} at 127.26.25.130:33995
I20240502 14:00:59.357625 3039 tablet_copy_service.cc:433] P 452ecbc085e54db49c3b01c20a96f860: ending tablet copy session c351a89a5fe34e788b9d8064fbffa751-638cb0f08414491ea80139eb51406dfe on tablet 638cb0f08414491ea80139eb51406dfe with peer c351a89a5fe34e788b9d8064fbffa751
I20240502 14:00:59.360563 32731 catalog_manager.cc:5484] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.361546 3017 raft_consensus.cc:1203] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:59.366946 3017 consensus_queue.cc:237] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:59.368703 405 raft_consensus.cc:1261] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.369123 2884 raft_consensus.cc:1261] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.369144 8276 consensus_queue.cc:1035] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.369580 5282 consensus_queue.cc:1035] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.372479 7995 raft_consensus.cc:2932] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.374157 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2ffed38329ca4e339a849c5ab3b09e8b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.374641 2886 raft_consensus.cc:2932] T 2ffed38329ca4e339a849c5ab3b09e8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.375294 405 raft_consensus.cc:2932] T 2ffed38329ca4e339a849c5ab3b09e8b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.377478 32726 catalog_manager.cc:5484] T 2ffed38329ca4e339a849c5ab3b09e8b P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.382711 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ffed38329ca4e339a849c5ab3b09e8b with delete_type TABLET_DATA_TOMBSTONED (TS 2b6796cf0e2e496e9dd641beb432f485 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:59.383347 8290 tablet_replica.cc:330] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:59.383508 8290 raft_consensus.cc:2229] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.383648 8290 raft_consensus.cc:2258] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.386224 8290 ts_tablet_manager.cc:1875] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.388420 8290 ts_tablet_manager.cc:1888] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.388491 8290 log.cc:1196] T 2ffed38329ca4e339a849c5ab3b09e8b P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/2ffed38329ca4e339a849c5ab3b09e8b
I20240502 14:00:59.389055 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 2ffed38329ca4e339a849c5ab3b09e8b (table rebalancer_extra_table_44 [id=6716820a9c7447f292faab28186dd3ae]) successfully deleted
I20240502 14:00:59.390132 8296 ts_tablet_manager.cc:902] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:59.390741 8296 tablet_copy_client.cc:308] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:00:59.391296 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 5e922f7cee8046e7b4263dd57b8d1185 from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:41861)
I20240502 14:00:59.391400 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 5e922f7cee8046e7b4263dd57b8d1185 from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:41861: session id = 478dfa53d7944f38b4e6cabee21b2d01-5e922f7cee8046e7b4263dd57b8d1185
I20240502 14:00:59.392359 2906 tablet_copy_source_session.cc:205] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:59.393816 8296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e922f7cee8046e7b4263dd57b8d1185. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:59.396636 8296 tablet_copy_client.cc:783] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:59.396803 8296 tablet_copy_client.cc:647] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:59.397941 8296 tablet_copy_client.cc:523] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:59.399257 8296 tablet_bootstrap.cc:492] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:59.402869 8296 tablet_bootstrap.cc:492] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:59.403126 8296 tablet_bootstrap.cc:492] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:00:59.403283 8296 ts_tablet_manager.cc:1367] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 14:00:59.403667 8296 raft_consensus.cc:348] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:59.403772 8296 raft_consensus.cc:729] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:00:59.403887 8296 consensus_queue.cc:260] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:59.404147 8296 ts_tablet_manager.cc:1398] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:59.405686 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-5e922f7cee8046e7b4263dd57b8d1185 received from {username='slave'} at 127.26.25.132:41861
I20240502 14:00:59.405759 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-5e922f7cee8046e7b4263dd57b8d1185 on tablet 5e922f7cee8046e7b4263dd57b8d1185 with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:00:59.410579 8159 raft_consensus.cc:1050] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485: attempting to promote NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 to VOTER
I20240502 14:00:59.410626 673 consensus_queue.cc:237] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:59.411002 8159 consensus_queue.cc:237] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:00:59.412029 2886 raft_consensus.cc:1261] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:59.412384 609 consensus_peers.cc:485] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 72590739196d48ac88d69b5bcd1124b0. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:59.412489 3017 raft_consensus.cc:1261] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:59.412487 405 raft_consensus.cc:1261] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.412487 5098 consensus_queue.cc:1035] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.412830 5098 consensus_queue.cc:1035] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.412956 6625 consensus_queue.cc:1035] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.413826 2884 raft_consensus.cc:1261] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.414397 8159 consensus_queue.cc:1035] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.415439 673 raft_consensus.cc:1261] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.415340 5098 raft_consensus.cc:2932] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:59.416363 6625 raft_consensus.cc:2932] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.416951 404 raft_consensus.cc:2932] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.418038 3017 raft_consensus.cc:2932] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:59.418844 8155 consensus_queue.cc:1035] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.420482 32729 catalog_manager.cc:5484] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.420579 32732 catalog_manager.cc:5484] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.421386 2885 raft_consensus.cc:2932] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.421913 673 raft_consensus.cc:2932] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.425890 2884 raft_consensus.cc:2932] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:59.438192 2884 consensus_queue.cc:237] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:59.439709 803 raft_consensus.cc:1261] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:59.439800 2819 consensus_peers.cc:485] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9e642731d7684e33b82eabf97c380123. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:59.439925 3017 raft_consensus.cc:1261] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:59.440178 7207 consensus_queue.cc:1035] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.440357 8167 consensus_queue.cc:1035] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.442282 7206 raft_consensus.cc:2932] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:59.442566 3017 raft_consensus.cc:2932] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:59.442724 803 raft_consensus.cc:2932] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:59.446519 32732 catalog_manager.cc:5484] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.449455 8053 raft_consensus.cc:1050] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:00:59.449934 8053 consensus_queue.cc:237] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:59.451778 803 raft_consensus.cc:1261] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.451906 3019 raft_consensus.cc:1261] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.452414 3150 raft_consensus.cc:1261] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.452684 8335 consensus_queue.cc:1035] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.453425 6625 raft_consensus.cc:1050] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485: attempting to promote NON_VOTER c351a89a5fe34e788b9d8064fbffa751 to VOTER
I20240502 14:00:59.453868 6625 consensus_queue.cc:237] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:59.453980 5715 consensus_queue.cc:1035] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.454342 5715 consensus_queue.cc:1035] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.455449 2884 raft_consensus.cc:1261] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.456043 3283 raft_consensus.cc:1261] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.456445 7006 consensus_queue.cc:1035] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.456779 7006 consensus_queue.cc:1035] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.457612 540 raft_consensus.cc:1261] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.458393 8159 consensus_queue.cc:1035] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.458679 672 consensus_queue.cc:237] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:00:59.460158 541 raft_consensus.cc:1261] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.460186 404 raft_consensus.cc:1261] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:59.460686 609 consensus_peers.cc:485] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): Couldn't send request to peer 4bcd7712aa844d98977a7d7e605c31f5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 987eabba58cd472696e64f30db468c71. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:59.460958 8257 consensus_queue.cc:1035] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.461230 7006 raft_consensus.cc:2932] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:59.461558 8257 consensus_queue.cc:1035] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.463225 3283 raft_consensus.cc:2932] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:59.464239 8257 raft_consensus.cc:2932] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:59.464682 540 raft_consensus.cc:2932] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:59.465616 32729 catalog_manager.cc:5484] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:59.467906 2884 raft_consensus.cc:2932] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:59.468722 32731 catalog_manager.cc:5484] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:59.471735 8335 raft_consensus.cc:2932] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.473349 540 raft_consensus.cc:2932] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:59.473740 3149 consensus_queue.cc:237] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:00:59.475364 3018 raft_consensus.cc:2932] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.475600 3283 raft_consensus.cc:1261] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.476025 8159 consensus_queue.cc:1035] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.477707 6625 raft_consensus.cc:2932] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:59.478128 3283 raft_consensus.cc:2932] T 49595e1db3c44669b438bd0ac1093a62 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:59.479215 803 raft_consensus.cc:2932] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.480090 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 49595e1db3c44669b438bd0ac1093a62 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.479354 32731 catalog_manager.cc:5484] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:59.484328 8167 raft_consensus.cc:1050] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 to VOTER
I20240502 14:00:59.484707 8167 consensus_queue.cc:237] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:00:59.484822 32728 catalog_manager.cc:5484] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.486340 3281 raft_consensus.cc:1261] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.486472 540 raft_consensus.cc:1261] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.486832 7206 consensus_queue.cc:1035] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.487419 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 49595e1db3c44669b438bd0ac1093a62 with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:59.487697 8222 tablet_replica.cc:330] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:59.487897 8222 raft_consensus.cc:2229] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.488025 8222 raft_consensus.cc:2258] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.488283 672 raft_consensus.cc:1261] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.488571 3018 raft_consensus.cc:1261] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.489066 7206 consensus_queue.cc:1035] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.489761 8159 consensus_queue.cc:1035] T 49595e1db3c44669b438bd0ac1093a62 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.489861 7206 consensus_queue.cc:1035] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.491084 540 raft_consensus.cc:2932] T 49595e1db3c44669b438bd0ac1093a62 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:00:59.493678 2884 consensus_queue.cc:237] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:59.494436 8222 ts_tablet_manager.cc:1875] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.495332 803 raft_consensus.cc:1261] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:59.495754 7206 consensus_queue.cc:1035] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.497519 8222 ts_tablet_manager.cc:1888] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.497550 3284 raft_consensus.cc:1261] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:59.497588 8222 log.cc:1196] T 49595e1db3c44669b438bd0ac1093a62 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/49595e1db3c44669b438bd0ac1093a62
I20240502 14:00:59.498087 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 49595e1db3c44669b438bd0ac1093a62 (table rebalancer_extra_table_50 [id=d90495f535a64ab390ffd317d36add87]) successfully deleted
I20240502 14:00:59.498284 7206 consensus_queue.cc:1035] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.500691 3150 raft_consensus.cc:2932] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
W20240502 14:00:59.502977 2822 consensus_peers.cc:485] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 00d3e2b37ce743bf90e211f5385efe90. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:59.504834 405 consensus_queue.cc:237] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:59.505811 7206 raft_consensus.cc:2932] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.506361 3150 raft_consensus.cc:1261] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.506994 7206 raft_consensus.cc:2932] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:59.507239 8053 consensus_queue.cc:1035] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.509883 8335 raft_consensus.cc:2932] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.511278 672 raft_consensus.cc:2932] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.511656 32720 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c1dc8816aa514d1abcce63ce0e0bf357 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.512689 3152 raft_consensus.cc:2932] T c1dc8816aa514d1abcce63ce0e0bf357 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.513970 803 raft_consensus.cc:2932] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:59.515566 3281 raft_consensus.cc:2932] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.516782 404 raft_consensus.cc:2932] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:00:59.517254 32734 catalog_manager.cc:5484] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.517382 32726 catalog_manager.cc:5484] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:59.518986 32732 catalog_manager.cc:5484] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.521688 2884 consensus_queue.cc:237] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:00:59.521898 3018 raft_consensus.cc:2932] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.523001 672 raft_consensus.cc:1261] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.523752 7207 consensus_queue.cc:1035] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.524045 3284 raft_consensus.cc:1261] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.524487 7207 consensus_queue.cc:1035] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.526223 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet c1dc8816aa514d1abcce63ce0e0bf357 with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:59.526695 8172 tablet_replica.cc:330] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:59.526832 8172 raft_consensus.cc:2229] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.526969 8172 raft_consensus.cc:2258] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.527971 8172 ts_tablet_manager.cc:1875] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.530172 8172 ts_tablet_manager.cc:1888] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.530239 8172 log.cc:1196] T c1dc8816aa514d1abcce63ce0e0bf357 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/c1dc8816aa514d1abcce63ce0e0bf357
I20240502 14:00:59.530344 8167 raft_consensus.cc:2932] T 2d9d248c2c6745489d2656752db9ddd5 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.531075 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet c1dc8816aa514d1abcce63ce0e0bf357 (table rebalancer_extra_table_36 [id=409fb32bfca44ecaa3f9d301ac14ce3a]) successfully deleted
I20240502 14:00:59.531672 803 raft_consensus.cc:1261] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.532007 672 raft_consensus.cc:2932] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.532326 3281 raft_consensus.cc:2932] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:00:59.532855 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2d9d248c2c6745489d2656752db9ddd5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.533737 8335 consensus_queue.cc:1035] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.534796 803 raft_consensus.cc:2932] T c1dc8816aa514d1abcce63ce0e0bf357 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.538053 32728 catalog_manager.cc:5484] T 2d9d248c2c6745489d2656752db9ddd5 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.543035 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d9d248c2c6745489d2656752db9ddd5 with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:00:59.539671 3281 raft_consensus.cc:2932] T 2d9d248c2c6745489d2656752db9ddd5 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.543318 8172 tablet_replica.cc:330] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:00:59.543493 8172 raft_consensus.cc:2229] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.543664 8172 raft_consensus.cc:2258] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.544456 8172 ts_tablet_manager.cc:1875] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.547261 8172 ts_tablet_manager.cc:1888] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.547489 8172 log.cc:1196] T 2d9d248c2c6745489d2656752db9ddd5 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/2d9d248c2c6745489d2656752db9ddd5
I20240502 14:00:59.548107 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 2d9d248c2c6745489d2656752db9ddd5 (table rebalancer_extra_table_4 [id=13b33238c8214cd6b88686b2b94abdb2]) successfully deleted
I20240502 14:00:59.550922 3281 consensus_queue.cc:237] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:59.552361 404 raft_consensus.cc:1261] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:59.552418 3152 raft_consensus.cc:1261] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:59.552703 3219 consensus_peers.cc:485] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81cd2e05295641678fa8d7711c797113. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:59.553050 7396 consensus_queue.cc:1035] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.553411 7615 consensus_queue.cc:1035] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.556074 672 raft_consensus.cc:1203] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:59.557008 7615 raft_consensus.cc:2932] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:59.559504 3150 raft_consensus.cc:2932] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:59.563658 32731 catalog_manager.cc:5484] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:59.564759 404 raft_consensus.cc:2932] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:00:59.651427 5098 raft_consensus.cc:1050] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:00:59.651877 5098 consensus_queue.cc:237] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:59.653610 3150 raft_consensus.cc:1261] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.653616 804 raft_consensus.cc:1261] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.654161 405 raft_consensus.cc:1261] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.655130 7982 consensus_queue.cc:1035] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.655961 6065 consensus_queue.cc:1035] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.656816 7982 consensus_queue.cc:1035] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.658921 7982 raft_consensus.cc:2932] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.659813 7396 raft_consensus.cc:1050] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:00:59.660312 7396 consensus_queue.cc:237] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:59.661752 405 raft_consensus.cc:2932] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.662019 404 raft_consensus.cc:1261] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.662410 7615 consensus_queue.cc:1035] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.663764 3150 raft_consensus.cc:1261] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.664535 3018 raft_consensus.cc:1261] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.664395 32731 catalog_manager.cc:5484] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.666308 8205 consensus_queue.cc:1035] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.666916 8205 consensus_queue.cc:1035] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.667153 804 raft_consensus.cc:2932] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.667205 3150 raft_consensus.cc:2932] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.673146 8205 raft_consensus.cc:2932] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.674397 3019 raft_consensus.cc:2932] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.678915 32731 catalog_manager.cc:5484] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.679260 672 consensus_queue.cc:237] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:00:59.683748 803 raft_consensus.cc:1261] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.684008 404 raft_consensus.cc:1261] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.684096 6065 consensus_queue.cc:1035] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.684437 6065 consensus_queue.cc:1035] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.685941 8257 raft_consensus.cc:2932] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.686498 404 raft_consensus.cc:2932] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.687690 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9a9aac11ac0b44339b6d9b0f84702871 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.687775 405 raft_consensus.cc:2932] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.690356 3152 raft_consensus.cc:2932] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.691251 32728 catalog_manager.cc:5484] T 9a9aac11ac0b44339b6d9b0f84702871 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.691329 3281 consensus_queue.cc:237] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:59.693336 405 raft_consensus.cc:1261] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.693668 8205 consensus_queue.cc:1035] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.694447 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a9aac11ac0b44339b6d9b0f84702871 with delete_type TABLET_DATA_TOMBSTONED (TS 2b6796cf0e2e496e9dd641beb432f485 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:59.694501 3019 raft_consensus.cc:1261] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.694624 8290 tablet_replica.cc:330] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:59.694736 8290 raft_consensus.cc:2229] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.694875 8290 raft_consensus.cc:2258] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.697209 8205 consensus_queue.cc:1035] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.698212 803 raft_consensus.cc:2932] T 9a9aac11ac0b44339b6d9b0f84702871 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:00:59.701300 8205 raft_consensus.cc:2932] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.702714 3019 raft_consensus.cc:2932] T 2f2c438f3a854f528ae8555cdb13721b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.703979 8290 ts_tablet_manager.cc:1875] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.706156 8290 ts_tablet_manager.cc:1888] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.706236 8290 log.cc:1196] T 9a9aac11ac0b44339b6d9b0f84702871 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/9a9aac11ac0b44339b6d9b0f84702871
I20240502 14:00:59.706816 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 9a9aac11ac0b44339b6d9b0f84702871 (table rebalancer_extra_table_50 [id=d90495f535a64ab390ffd317d36add87]) successfully deleted
I20240502 14:00:59.707775 8257 raft_consensus.cc:1050] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER da3f8160557e454f8547f45e80c2f013 to VOTER
I20240502 14:00:59.707947 405 raft_consensus.cc:2932] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.708202 8257 consensus_queue.cc:237] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:59.710006 540 raft_consensus.cc:1261] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:59.710331 32719 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2f2c438f3a854f528ae8555cdb13721b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.710749 8244 consensus_queue.cc:1035] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:59.712114 803 raft_consensus.cc:1261] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:59.712631 8244 consensus_queue.cc:1035] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:59.713222 32728 catalog_manager.cc:5484] T 2f2c438f3a854f528ae8555cdb13721b P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.715656 3281 raft_consensus.cc:1261] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:59.715682 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f2c438f3a854f528ae8555cdb13721b with delete_type TABLET_DATA_TOMBSTONED (TS 2b6796cf0e2e496e9dd641beb432f485 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:59.716080 7982 consensus_queue.cc:1035] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:59.718194 8290 tablet_replica.cc:330] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:59.718356 8290 raft_consensus.cc:2229] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.718511 8290 raft_consensus.cc:2258] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.719062 8290 ts_tablet_manager.cc:1875] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.720482 7982 raft_consensus.cc:2932] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.721117 8290 ts_tablet_manager.cc:1888] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.721253 8290 log.cc:1196] T 2f2c438f3a854f528ae8555cdb13721b P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/2f2c438f3a854f528ae8555cdb13721b
I20240502 14:00:59.721841 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 2f2c438f3a854f528ae8555cdb13721b (table rebalancer_extra_table_22 [id=de113df849174bb5a42c2527f02cac0d]) successfully deleted
I20240502 14:00:59.723263 540 raft_consensus.cc:2932] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.723526 3281 raft_consensus.cc:2932] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.723802 803 raft_consensus.cc:2932] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.723982 32732 catalog_manager.cc:5484] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.733687 672 consensus_queue.cc:237] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:00:59.735206 3283 raft_consensus.cc:1261] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:59.735538 803 raft_consensus.cc:1261] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:59.735924 8257 consensus_queue.cc:1035] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:59.736238 8257 consensus_queue.cc:1035] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:59.738034 8244 raft_consensus.cc:2932] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.738559 803 raft_consensus.cc:2932] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.742167 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 567df9d664b7423785175fa385f83e53 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.742461 3283 raft_consensus.cc:2932] T 567df9d664b7423785175fa385f83e53 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.742858 32732 catalog_manager.cc:5484] T 567df9d664b7423785175fa385f83e53 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:00:59.745563 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 567df9d664b7423785175fa385f83e53 with delete_type TABLET_DATA_TOMBSTONED (TS c351a89a5fe34e788b9d8064fbffa751 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60890
I20240502 14:00:59.746965 8160 tablet_replica.cc:330] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:00:59.747126 8160 raft_consensus.cc:2229] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.747274 8160 raft_consensus.cc:2258] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.749173 8160 ts_tablet_manager.cc:1875] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.753840 8160 ts_tablet_manager.cc:1888] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:59.753991 8160 log.cc:1196] T 567df9d664b7423785175fa385f83e53 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/567df9d664b7423785175fa385f83e53
I20240502 14:00:59.754534 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 567df9d664b7423785175fa385f83e53 (table rebalance_test_table_13 [id=8dc2f81034eb43978338637236c345c6]) successfully deleted
I20240502 14:00:59.765516 540 raft_consensus.cc:1203] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:59.796716 2884 consensus_queue.cc:237] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:59.798369 405 raft_consensus.cc:1261] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:59.798403 3019 raft_consensus.cc:1261] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:59.798393 2819 consensus_peers.cc:485] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ec443abbb4954bf3b28336e0b5bc0c4b. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:59.798764 8358 consensus_queue.cc:1035] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.798949 7207 consensus_queue.cc:1035] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.800977 8167 raft_consensus.cc:2932] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:59.801362 405 raft_consensus.cc:2932] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:59.801451 3019 raft_consensus.cc:2932] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:59.804638 32732 catalog_manager.cc:5484] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.818140 3283 consensus_queue.cc:237] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:00:59.819638 3019 raft_consensus.cc:1261] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:59.819762 2884 raft_consensus.cc:1261] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:59.819800 3216 consensus_peers.cc:485] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be6290bc7df6465f8a43aae431b34f8b. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:59.820052 8426 consensus_queue.cc:1035] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.820222 8205 consensus_queue.cc:1035] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.822335 8426 raft_consensus.cc:2932] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:59.822810 3019 raft_consensus.cc:2932] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:59.826177 2884 raft_consensus.cc:2932] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:59.826812 32731 catalog_manager.cc:5484] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:00:59.835453 2884 consensus_queue.cc:237] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:59.836915 672 raft_consensus.cc:1261] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:59.837002 3152 raft_consensus.cc:1261] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:59.837208 2819 consensus_peers.cc:485] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): Couldn't send request to peer 452ecbc085e54db49c3b01c20a96f860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 79db2b3bbcb041f6be0d86b150a3d324. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:59.837359 8167 consensus_queue.cc:1035] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.837554 6939 consensus_queue.cc:1035] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.839282 7207 raft_consensus.cc:2932] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:59.839732 3152 raft_consensus.cc:2932] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:59.840263 672 raft_consensus.cc:2932] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:59.843322 32732 catalog_manager.cc:5484] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:59.845901 8296 ts_tablet_manager.cc:902] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:59.846408 8296 tablet_copy_client.cc:308] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:59.846868 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 72590739196d48ac88d69b5bcd1124b0 from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:56893)
I20240502 14:00:59.846961 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 72590739196d48ac88d69b5bcd1124b0 from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:56893: session id = 478dfa53d7944f38b4e6cabee21b2d01-72590739196d48ac88d69b5bcd1124b0
I20240502 14:00:59.848042 693 tablet_copy_source_session.cc:205] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:59.848935 8296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72590739196d48ac88d69b5bcd1124b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:59.851289 8296 tablet_copy_client.cc:783] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:59.851433 8296 tablet_copy_client.cc:647] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:59.852890 8296 tablet_copy_client.cc:523] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:59.853039 2884 consensus_queue.cc:237] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:00:59.854113 672 raft_consensus.cc:1261] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.854211 405 raft_consensus.cc:1261] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.854503 6939 consensus_queue.cc:1035] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.854548 8296 tablet_bootstrap.cc:492] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:00:59.855751 6939 consensus_queue.cc:1035] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.856657 6939 raft_consensus.cc:2932] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.857338 672 raft_consensus.cc:2932] T 8cf2d71983204c3c96d112cab1004478 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.858839 8296 tablet_bootstrap.cc:492] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:59.859066 8296 tablet_bootstrap.cc:492] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:00:59.859205 8296 ts_tablet_manager.cc:1367] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20240502 14:00:59.859624 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8cf2d71983204c3c96d112cab1004478 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.859632 8296 raft_consensus.cc:348] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:59.859742 8296 raft_consensus.cc:729] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:00:59.859858 8296 consensus_queue.cc:260] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:00:59.860214 8296 ts_tablet_manager.cc:1398] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:59.861366 32731 catalog_manager.cc:5484] T 8cf2d71983204c3c96d112cab1004478 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.862205 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-72590739196d48ac88d69b5bcd1124b0 received from {username='slave'} at 127.26.25.132:56893
I20240502 14:00:59.862272 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-72590739196d48ac88d69b5bcd1124b0 on tablet 72590739196d48ac88d69b5bcd1124b0 with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:00:59.862406 405 raft_consensus.cc:2932] T 8cf2d71983204c3c96d112cab1004478 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:00:59.867794 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cf2d71983204c3c96d112cab1004478 with delete_type TABLET_DATA_TOMBSTONED (TS 2b6796cf0e2e496e9dd641beb432f485 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60258
I20240502 14:00:59.868232 8290 tablet_replica.cc:330] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:00:59.868379 8290 raft_consensus.cc:2229] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.868525 8290 raft_consensus.cc:2258] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.869262 8290 ts_tablet_manager.cc:1875] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.871568 8290 ts_tablet_manager.cc:1888] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.871630 8290 log.cc:1196] T 8cf2d71983204c3c96d112cab1004478 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/8cf2d71983204c3c96d112cab1004478
I20240502 14:00:59.872310 672 consensus_queue.cc:237] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:00:59.873555 804 raft_consensus.cc:1261] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:59.873919 607 consensus_peers.cc:485] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08f30aab835c4380b8b7a01400e5aa55. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:59.874092 7982 consensus_queue.cc:1035] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.875690 405 raft_consensus.cc:1261] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.875725 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 8cf2d71983204c3c96d112cab1004478 (table rebalancer_extra_table_28 [id=8bbaf40cee36492e87629b1979f62041]) successfully deleted
I20240502 14:00:59.876077 7982 consensus_queue.cc:1035] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.876417 8257 raft_consensus.cc:2932] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:59.877008 405 raft_consensus.cc:2932] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:59.878023 804 raft_consensus.cc:2932] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:00:59.880661 32728 catalog_manager.cc:5484] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.887073 8519 ts_tablet_manager.cc:902] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:59.887585 8519 tablet_copy_client.cc:308] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:00:59.888085 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 00d3e2b37ce743bf90e211f5385efe90 from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:33043)
I20240502 14:00:59.888195 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 00d3e2b37ce743bf90e211f5385efe90 from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:33043: session id = 2b6796cf0e2e496e9dd641beb432f485-00d3e2b37ce743bf90e211f5385efe90
I20240502 14:00:59.889122 2906 tablet_copy_source_session.cc:205] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:59.890017 8519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00d3e2b37ce743bf90e211f5385efe90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:59.893105 8519 tablet_copy_client.cc:783] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:59.893323 8519 tablet_copy_client.cc:647] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:59.894145 804 consensus_queue.cc:237] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:00:59.894464 8519 tablet_copy_client.cc:523] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:59.895691 3283 raft_consensus.cc:1261] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:59.895889 405 raft_consensus.cc:1261] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:59.896106 738 consensus_peers.cc:485] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9185ea12db4b413a8c76800b828836da. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:59.896140 6174 consensus_queue.cc:1035] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.896620 6174 consensus_queue.cc:1035] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.898316 8519 tablet_bootstrap.cc:492] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:00:59.898576 6174 raft_consensus.cc:2932] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:59.900853 405 raft_consensus.cc:2932] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:59.904583 32728 catalog_manager.cc:5484] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.908484 8528 ts_tablet_manager.cc:902] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:00:59.909274 8528 tablet_copy_client.cc:308] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:00:59.910069 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 987eabba58cd472696e64f30db468c71 from peer 4bcd7712aa844d98977a7d7e605c31f5 ({username='slave'} at 127.26.25.133:46031)
I20240502 14:00:59.910171 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 987eabba58cd472696e64f30db468c71 from peer 4bcd7712aa844d98977a7d7e605c31f5 at {username='slave'} at 127.26.25.133:46031: session id = 4bcd7712aa844d98977a7d7e605c31f5-987eabba58cd472696e64f30db468c71
I20240502 14:00:59.911274 693 tablet_copy_source_session.cc:205] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:59.912711 8528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987eabba58cd472696e64f30db468c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:59.913475 3283 raft_consensus.cc:2932] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:00:59.914835 8528 tablet_copy_client.cc:783] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:59.914995 8528 tablet_copy_client.cc:647] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:59.916703 8519 tablet_bootstrap.cc:492] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:59.916966 8528 tablet_copy_client.cc:523] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:59.917124 8519 tablet_bootstrap.cc:492] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:00:59.917352 8519 ts_tablet_manager.cc:1367] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20240502 14:00:59.918423 8519 raft_consensus.cc:348] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:59.918557 8519 raft_consensus.cc:729] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:00:59.918731 8528 tablet_bootstrap.cc:492] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:00:59.918990 8519 consensus_queue.cc:260] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:00:59.919404 8519 ts_tablet_manager.cc:1398] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:59.921151 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-00d3e2b37ce743bf90e211f5385efe90 received from {username='slave'} at 127.26.25.135:33043
I20240502 14:00:59.921222 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-00d3e2b37ce743bf90e211f5385efe90 on tablet 00d3e2b37ce743bf90e211f5385efe90 with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:00:59.922175 2884 consensus_queue.cc:237] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:00:59.923969 3283 raft_consensus.cc:1261] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:00:59.925122 2819 consensus_peers.cc:485] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): Couldn't send request to peer 452ecbc085e54db49c3b01c20a96f860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c27c9fb983c49a59a2f84776775e1bd. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:59.925247 7206 consensus_queue.cc:1035] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.925558 3152 raft_consensus.cc:1261] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:59.926642 7206 consensus_queue.cc:1035] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:59.928859 8358 raft_consensus.cc:2932] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:59.931123 3152 consensus_queue.cc:237] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:00:59.931871 3150 raft_consensus.cc:2932] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:59.932483 405 raft_consensus.cc:1261] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.932917 672 raft_consensus.cc:1261] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.932902 6625 consensus_queue.cc:1035] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.934237 32731 catalog_manager.cc:5484] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:59.934831 7246 consensus_queue.cc:1035] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.935139 7246 raft_consensus.cc:2932] T a4015371eba240cb8e8f7e2180ac3d27 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.937775 405 raft_consensus.cc:2932] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.939790 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a4015371eba240cb8e8f7e2180ac3d27 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.942016 32728 catalog_manager.cc:5484] T a4015371eba240cb8e8f7e2180ac3d27 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.944989 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet a4015371eba240cb8e8f7e2180ac3d27 with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:00:59.945202 8222 tablet_replica.cc:330] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:00:59.945343 8222 raft_consensus.cc:2229] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.945324 3283 raft_consensus.cc:2932] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:00:59.945487 8222 raft_consensus.cc:2258] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.946228 8222 ts_tablet_manager.cc:1875] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.948410 8222 ts_tablet_manager.cc:1888] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.948473 8222 log.cc:1196] T a4015371eba240cb8e8f7e2180ac3d27 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/a4015371eba240cb8e8f7e2180ac3d27
I20240502 14:00:59.950395 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet a4015371eba240cb8e8f7e2180ac3d27 (table rebalancer_extra_table_12 [id=4e2ce13a9a4c4a03a79ac1f8bfba20bc]) successfully deleted
I20240502 14:00:59.952759 8528 tablet_bootstrap.cc:492] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:59.953042 8528 tablet_bootstrap.cc:492] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap complete.
I20240502 14:00:59.953119 7615 raft_consensus.cc:1050] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013: attempting to promote NON_VOTER 452ecbc085e54db49c3b01c20a96f860 to VOTER
I20240502 14:00:59.953215 8528 ts_tablet_manager.cc:1367] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.035s user 0.020s sys 0.008s
I20240502 14:00:59.953531 7615 consensus_queue.cc:237] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:00:59.953657 8528 raft_consensus.cc:348] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:00:59.953759 8528 raft_consensus.cc:729] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: LEARNER
I20240502 14:00:59.953852 8528 consensus_queue.cc:260] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:00:59.954141 8528 ts_tablet_manager.cc:1398] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:59.954413 804 raft_consensus.cc:1203] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:59.955090 672 raft_consensus.cc:1261] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.955142 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-987eabba58cd472696e64f30db468c71 received from {username='slave'} at 127.26.25.133:46031
I20240502 14:00:59.955207 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-987eabba58cd472696e64f30db468c71 on tablet 987eabba58cd472696e64f30db468c71 with peer 4bcd7712aa844d98977a7d7e605c31f5
I20240502 14:00:59.955565 8367 consensus_queue.cc:1035] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.956038 3018 raft_consensus.cc:1261] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.956528 8367 consensus_queue.cc:1035] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.957661 2884 raft_consensus.cc:1261] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:59.958163 672 raft_consensus.cc:2932] T a4015371eba240cb8e8f7e2180ac3d27 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:00:59.958369 8367 consensus_queue.cc:1035] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:59.962052 7766 raft_consensus.cc:2932] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.965534 3018 raft_consensus.cc:2932] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.965587 672 raft_consensus.cc:2932] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.968659 32732 catalog_manager.cc:5484] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:59.974011 2885 raft_consensus.cc:2932] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.978335 3283 consensus_queue.cc:237] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:00:59.979610 3019 raft_consensus.cc:1261] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.980020 8367 consensus_queue.cc:1035] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.980106 2885 raft_consensus.cc:1261] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:59.980734 8367 consensus_queue.cc:1035] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:59.982053 8205 raft_consensus.cc:2932] T f51e005446c64a07b18f0dbf5056afbc P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.983156 2884 raft_consensus.cc:2932] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.984054 3019 raft_consensus.cc:2932] T f51e005446c64a07b18f0dbf5056afbc P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.984602 32719 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f51e005446c64a07b18f0dbf5056afbc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:59.987413 32732 catalog_manager.cc:5484] T f51e005446c64a07b18f0dbf5056afbc P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) evicted. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:00:59.994011 653 tablet_service.cc:1514] Processing DeleteTablet for tablet f51e005446c64a07b18f0dbf5056afbc with delete_type TABLET_DATA_TOMBSTONED (TS 58107b611f8f475a910a6fd54dfd1fb9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54104
I20240502 14:00:59.994377 8568 tablet_replica.cc:330] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9: stopping tablet replica
I20240502 14:00:59.994537 8568 raft_consensus.cc:2229] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:59.994695 8568 raft_consensus.cc:2258] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:59.995880 8568 ts_tablet_manager.cc:1875] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:59.998220 8568 ts_tablet_manager.cc:1888] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:59.998308 8568 log.cc:1196] T f51e005446c64a07b18f0dbf5056afbc P 58107b611f8f475a910a6fd54dfd1fb9: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/wals/f51e005446c64a07b18f0dbf5056afbc
I20240502 14:00:59.998327 8573 ts_tablet_manager.cc:902] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:00:59.998831 8573 tablet_copy_client.cc:308] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:00:59.998911 32718 catalog_manager.cc:4830] TS 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): tablet f51e005446c64a07b18f0dbf5056afbc (table rebalancer_extra_table_60 [id=4bef3bba6d9447d8a24755864aec22c2]) successfully deleted
I20240502 14:00:59.999241 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 9e642731d7684e33b82eabf97c380123 from peer 58107b611f8f475a910a6fd54dfd1fb9 ({username='slave'} at 127.26.25.131:52409)
I20240502 14:00:59.999351 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 9e642731d7684e33b82eabf97c380123 from peer 58107b611f8f475a910a6fd54dfd1fb9 at {username='slave'} at 127.26.25.131:52409: session id = 58107b611f8f475a910a6fd54dfd1fb9-9e642731d7684e33b82eabf97c380123
I20240502 14:01:00.000159 2906 tablet_copy_source_session.cc:205] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.000946 8573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e642731d7684e33b82eabf97c380123. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.002900 8573 tablet_copy_client.cc:783] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.003049 8573 tablet_copy_client.cc:647] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.004412 8573 tablet_copy_client.cc:523] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.005685 8573 tablet_bootstrap.cc:492] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:01:00.009219 8573 tablet_bootstrap.cc:492] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.009442 8573 tablet_bootstrap.cc:492] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap complete.
I20240502 14:01:00.009596 8573 ts_tablet_manager.cc:1367] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20240502 14:01:00.009997 8573 raft_consensus.cc:348] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:00.010114 8573 raft_consensus.cc:729] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: LEARNER
I20240502 14:01:00.010210 8573 consensus_queue.cc:260] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:00.010488 8573 ts_tablet_manager.cc:1398] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:00.013325 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-9e642731d7684e33b82eabf97c380123 received from {username='slave'} at 127.26.25.131:52409
I20240502 14:01:00.013417 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-9e642731d7684e33b82eabf97c380123 on tablet 9e642731d7684e33b82eabf97c380123 with peer 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:01:00.088066 7396 raft_consensus.cc:1050] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: attempting to promote NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 to VOTER
I20240502 14:01:00.088521 7396 consensus_queue.cc:237] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:00.089890 2884 raft_consensus.cc:1261] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.089962 541 raft_consensus.cc:1261] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.090482 672 raft_consensus.cc:1261] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.090502 8367 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.090858 8205 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.091024 8426 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.094985 8426 raft_consensus.cc:2932] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:00.095503 673 raft_consensus.cc:2932] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:00.095599 2884 raft_consensus.cc:2932] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:00.097808 541 raft_consensus.cc:2932] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:00.098604 32728 catalog_manager.cc:5484] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:00.149206 541 consensus_queue.cc:237] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:01:00.150210 8296 ts_tablet_manager.cc:902] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:01:00.150697 3019 raft_consensus.cc:1261] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.150712 3152 raft_consensus.cc:1261] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.150758 8296 tablet_copy_client.cc:308] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.136:43131
I20240502 14:01:00.151247 8662 consensus_queue.cc:1035] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:00.151286 3304 tablet_copy_service.cc:140] P da3f8160557e454f8547f45e80c2f013: Received BeginTabletCopySession request for tablet 81cd2e05295641678fa8d7711c797113 from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:51927)
I20240502 14:01:00.151391 3304 tablet_copy_service.cc:161] P da3f8160557e454f8547f45e80c2f013: Beginning new tablet copy session on tablet 81cd2e05295641678fa8d7711c797113 from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:51927: session id = 478dfa53d7944f38b4e6cabee21b2d01-81cd2e05295641678fa8d7711c797113
I20240502 14:01:00.151649 8662 consensus_queue.cc:1035] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:00.152484 3304 tablet_copy_source_session.cc:205] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013: Tablet Copy: opened 0 blocks and 1 log segments
W20240502 14:01:00.152901 476 consensus_peers.cc:485] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2ce30696a474d23818c3f70cc86e9dd. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:00.153492 8296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81cd2e05295641678fa8d7711c797113. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.155174 7952 raft_consensus.cc:2932] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:01:00.155733 8296 tablet_copy_client.cc:783] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.155948 8296 tablet_copy_client.cc:647] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.155858 3152 raft_consensus.cc:2932] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:01:00.156853 3018 raft_consensus.cc:2932] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:01:00.158732 8296 tablet_copy_client.cc:523] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.160085 8296 tablet_bootstrap.cc:492] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:01:00.160703 8275 raft_consensus.cc:1050] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860: attempting to promote NON_VOTER c351a89a5fe34e788b9d8064fbffa751 to VOTER
I20240502 14:01:00.159685 32732 catalog_manager.cc:5484] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:01:00.161136 8275 consensus_queue.cc:237] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:00.162492 541 raft_consensus.cc:1261] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.162933 804 raft_consensus.cc:1261] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.163522 8276 consensus_queue.cc:1035] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.165346 8276 consensus_queue.cc:1035] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.166129 405 raft_consensus.cc:1261] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.167649 8276 consensus_queue.cc:1035] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.168608 8276 raft_consensus.cc:2932] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:00.169891 8296 tablet_bootstrap.cc:492] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.170145 8296 tablet_bootstrap.cc:492] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:01:00.170293 8296 ts_tablet_manager.cc:1367] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20240502 14:01:00.170722 8296 raft_consensus.cc:348] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:01:00.170832 8296 raft_consensus.cc:729] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:01:00.170925 8296 consensus_queue.cc:260] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:01:00.171245 8296 ts_tablet_manager.cc:1398] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 14:01:00.172320 32732 catalog_manager.cc:5484] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:00.173520 541 raft_consensus.cc:2932] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:00.177143 804 raft_consensus.cc:2932] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:00.177474 3304 tablet_copy_service.cc:342] P da3f8160557e454f8547f45e80c2f013: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-81cd2e05295641678fa8d7711c797113 received from {username='slave'} at 127.26.25.132:51927
I20240502 14:01:00.177556 3304 tablet_copy_service.cc:433] P da3f8160557e454f8547f45e80c2f013: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-81cd2e05295641678fa8d7711c797113 on tablet 81cd2e05295641678fa8d7711c797113 with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:00.177423 405 raft_consensus.cc:2932] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:00.179464 3018 consensus_queue.cc:237] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:00.181416 2884 raft_consensus.cc:1261] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.182108 8670 consensus_queue.cc:1035] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:01:00.182560 2951 consensus_peers.cc:485] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 10c70369a07e45afb7bd885b58daec40. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:00.183261 3018 consensus_queue.cc:237] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:00.184553 541 raft_consensus.cc:1261] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:00.184840 8276 raft_consensus.cc:2932] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:00.185019 803 raft_consensus.cc:1261] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:00.185003 8275 consensus_queue.cc:1035] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:00.185961 8275 consensus_queue.cc:1035] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:00.186219 2884 raft_consensus.cc:2932] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:00.186731 5282 raft_consensus.cc:2932] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:00.187187 804 raft_consensus.cc:1261] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.187737 8275 consensus_queue.cc:1035] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:00.188094 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 638cb0f08414491ea80139eb51406dfe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:00.188376 541 raft_consensus.cc:2932] T 638cb0f08414491ea80139eb51406dfe P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:00.189903 32729 catalog_manager.cc:5484] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:00.190577 803 raft_consensus.cc:2932] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:00.192445 804 raft_consensus.cc:2932] T 638cb0f08414491ea80139eb51406dfe P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:00.196115 32729 catalog_manager.cc:5484] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:00.198704 387 tablet_service.cc:1514] Processing DeleteTablet for tablet 638cb0f08414491ea80139eb51406dfe with delete_type TABLET_DATA_TOMBSTONED (TS f003b732dd0f46958262b42f8d20305e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43314
I20240502 14:01:00.201243 8688 tablet_replica.cc:330] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:01:00.201417 8688 raft_consensus.cc:2229] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:00.201629 8688 raft_consensus.cc:2258] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:00.203393 8688 ts_tablet_manager.cc:1875] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:00.206533 2884 consensus_queue.cc:237] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:01:00.207754 8688 ts_tablet_manager.cc:1888] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:00.207866 8688 log.cc:1196] T 638cb0f08414491ea80139eb51406dfe P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/638cb0f08414491ea80139eb51406dfe
I20240502 14:01:00.208384 3018 raft_consensus.cc:1261] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.208410 3152 raft_consensus.cc:1261] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.208783 7206 consensus_queue.cc:1035] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:00.209045 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 638cb0f08414491ea80139eb51406dfe (table rebalancer_extra_table_12 [id=4e2ce13a9a4c4a03a79ac1f8bfba20bc]) successfully deleted
I20240502 14:01:00.209228 6939 consensus_queue.cc:1035] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:01:00.210232 2822 consensus_peers.cc:485] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f3d9ae2ed9a84c2c928dc1d03f325fa3. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:00.210714 7206 raft_consensus.cc:2932] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:00.211227 3152 raft_consensus.cc:2932] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:00.212363 3018 raft_consensus.cc:2932] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:00.214934 32729 catalog_manager.cc:5484] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:00.226465 804 consensus_queue.cc:237] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:01:00.227962 541 raft_consensus.cc:1261] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:01:00.228085 739 consensus_peers.cc:485] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 167e1eb3a4034f3ca75e721b8140d757. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:00.228142 3152 raft_consensus.cc:1261] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.228323 6174 consensus_queue.cc:1035] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:00.228564 8525 consensus_queue.cc:1035] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:00.230279 8161 raft_consensus.cc:2932] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:01:00.230757 3152 raft_consensus.cc:2932] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:01:00.230806 541 raft_consensus.cc:2932] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:01:00.234328 32731 catalog_manager.cc:5484] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:01:00.249646 405 consensus_queue.cc:237] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:00.251340 3152 raft_consensus.cc:1261] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.251677 3018 raft_consensus.cc:1261] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.251833 8334 consensus_queue.cc:1035] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:01:00.252027 341 consensus_peers.cc:485] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 604579a2517d4c2eaf855db299d15d5e. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:00.252341 8334 consensus_queue.cc:1035] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:00.253719 8615 raft_consensus.cc:2932] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:00.254182 3152 raft_consensus.cc:2932] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:00.255162 3018 raft_consensus.cc:2932] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:00.258082 32728 catalog_manager.cc:5484] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:00.286374 8728 ts_tablet_manager.cc:902] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751: Initiating tablet copy from peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:00.287022 8728 tablet_copy_client.cc:308] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.132:33443
I20240502 14:01:00.287806 826 tablet_copy_service.cc:140] P 478dfa53d7944f38b4e6cabee21b2d01: Received BeginTabletCopySession request for tablet 9185ea12db4b413a8c76800b828836da from peer c351a89a5fe34e788b9d8064fbffa751 ({username='slave'} at 127.26.25.130:33155)
I20240502 14:01:00.287947 826 tablet_copy_service.cc:161] P 478dfa53d7944f38b4e6cabee21b2d01: Beginning new tablet copy session on tablet 9185ea12db4b413a8c76800b828836da from peer c351a89a5fe34e788b9d8064fbffa751 at {username='slave'} at 127.26.25.130:33155: session id = c351a89a5fe34e788b9d8064fbffa751-9185ea12db4b413a8c76800b828836da
I20240502 14:01:00.288903 826 tablet_copy_source_session.cc:205] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.289824 8728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9185ea12db4b413a8c76800b828836da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.292493 8728 tablet_copy_client.cc:783] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.292670 8728 tablet_copy_client.cc:647] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.293771 8728 tablet_copy_client.cc:523] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.295209 8728 tablet_bootstrap.cc:492] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:01:00.299697 8728 tablet_bootstrap.cc:492] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.300019 8728 tablet_bootstrap.cc:492] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751: Bootstrap complete.
I20240502 14:01:00.300215 8728 ts_tablet_manager.cc:1367] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 14:01:00.300745 8728 raft_consensus.cc:348] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:01:00.300897 8728 raft_consensus.cc:729] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: LEARNER
I20240502 14:01:00.301071 8728 consensus_queue.cc:260] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:01:00.301438 8728 ts_tablet_manager.cc:1398] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:00.301877 826 tablet_copy_service.cc:342] P 478dfa53d7944f38b4e6cabee21b2d01: Request end of tablet copy session c351a89a5fe34e788b9d8064fbffa751-9185ea12db4b413a8c76800b828836da received from {username='slave'} at 127.26.25.130:33155
I20240502 14:01:00.301946 826 tablet_copy_service.cc:433] P 478dfa53d7944f38b4e6cabee21b2d01: ending tablet copy session c351a89a5fe34e788b9d8064fbffa751-9185ea12db4b413a8c76800b828836da on tablet 9185ea12db4b413a8c76800b828836da with peer c351a89a5fe34e788b9d8064fbffa751
I20240502 14:01:00.353464 8728 ts_tablet_manager.cc:902] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:01:00.353914 8728 tablet_copy_client.cc:308] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:01:00.354370 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet ec443abbb4954bf3b28336e0b5bc0c4b from peer c351a89a5fe34e788b9d8064fbffa751 ({username='slave'} at 127.26.25.130:46819)
I20240502 14:01:00.354502 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet ec443abbb4954bf3b28336e0b5bc0c4b from peer c351a89a5fe34e788b9d8064fbffa751 at {username='slave'} at 127.26.25.130:46819: session id = c351a89a5fe34e788b9d8064fbffa751-ec443abbb4954bf3b28336e0b5bc0c4b
I20240502 14:01:00.355687 2906 tablet_copy_source_session.cc:205] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.355799 8788 ts_tablet_manager.cc:902] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:01:00.356460 8788 tablet_copy_client.cc:308] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:01:00.356897 8728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec443abbb4954bf3b28336e0b5bc0c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.356889 2906 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 79db2b3bbcb041f6be0d86b150a3d324 from peer 452ecbc085e54db49c3b01c20a96f860 ({username='slave'} at 127.26.25.134:33823)
I20240502 14:01:00.356973 2906 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 79db2b3bbcb041f6be0d86b150a3d324 from peer 452ecbc085e54db49c3b01c20a96f860 at {username='slave'} at 127.26.25.134:33823: session id = 452ecbc085e54db49c3b01c20a96f860-79db2b3bbcb041f6be0d86b150a3d324
I20240502 14:01:00.357676 2906 tablet_copy_source_session.cc:205] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.358511 8788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79db2b3bbcb041f6be0d86b150a3d324. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.359494 8728 tablet_copy_client.cc:783] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.360096 8728 tablet_copy_client.cc:647] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.360251 7206 raft_consensus.cc:1050] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:01:00.360694 7206 consensus_queue.cc:237] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:01:00.360909 8788 tablet_copy_client.cc:783] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.361075 8788 tablet_copy_client.cc:647] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.361366 8728 tablet_copy_client.cc:523] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.362236 3283 raft_consensus.cc:1261] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.362336 803 raft_consensus.cc:1261] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.362718 7206 consensus_queue.cc:1035] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.362776 8728 tablet_bootstrap.cc:492] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:01:00.362882 8167 consensus_queue.cc:1035] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.364284 3018 raft_consensus.cc:1261] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.365134 6939 consensus_queue.cc:1035] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.368183 8167 raft_consensus.cc:2932] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.368531 3283 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b20d05c7433e4f56832233e2fc9adc84"
dest_uuid: "da3f8160557e454f8547f45e80c2f013"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46336
I20240502 14:01:00.368649 3283 raft_consensus.cc:595] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:01:00.368654 3284 raft_consensus.cc:2932] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.370188 803 raft_consensus.cc:2932] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.372411 32732 catalog_manager.cc:5484] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:00.372459 3018 raft_consensus.cc:2932] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.374963 8728 tablet_bootstrap.cc:492] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.375228 8728 tablet_bootstrap.cc:492] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751: Bootstrap complete.
I20240502 14:01:00.375402 8728 ts_tablet_manager.cc:1367] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.003s
I20240502 14:01:00.377575 8728 raft_consensus.cc:348] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:01:00.379356 8728 raft_consensus.cc:729] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: LEARNER
I20240502 14:01:00.379313 2884 consensus_queue.cc:237] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:01:00.379482 8728 consensus_queue.cc:260] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:01:00.380558 3018 raft_consensus.cc:1261] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:00.381186 6939 consensus_queue.cc:1035] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:00.381668 803 raft_consensus.cc:1261] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:00.382133 8358 consensus_queue.cc:1035] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:00.382282 8728 ts_tablet_manager.cc:1398] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.007s user 0.001s sys 0.002s
I20240502 14:01:00.383294 8803 ts_tablet_manager.cc:902] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:01:00.383697 2906 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session c351a89a5fe34e788b9d8064fbffa751-ec443abbb4954bf3b28336e0b5bc0c4b received from {username='slave'} at 127.26.25.130:46819
I20240502 14:01:00.383772 2906 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session c351a89a5fe34e788b9d8064fbffa751-ec443abbb4954bf3b28336e0b5bc0c4b on tablet ec443abbb4954bf3b28336e0b5bc0c4b with peer c351a89a5fe34e788b9d8064fbffa751
I20240502 14:01:00.384222 8803 tablet_copy_client.cc:308] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:01:00.384644 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 08f30aab835c4380b8b7a01400e5aa55 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:47487)
I20240502 14:01:00.384744 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 08f30aab835c4380b8b7a01400e5aa55 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:47487: session id = da3f8160557e454f8547f45e80c2f013-08f30aab835c4380b8b7a01400e5aa55
I20240502 14:01:00.384791 803 raft_consensus.cc:1203] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:00.385128 8788 tablet_copy_client.cc:523] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.385697 693 tablet_copy_source_session.cc:205] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.386422 8788 tablet_bootstrap.cc:492] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:01:00.386608 8803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f30aab835c4380b8b7a01400e5aa55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.389189 8803 tablet_copy_client.cc:783] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.389346 8803 tablet_copy_client.cc:647] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.390273 673 raft_consensus.cc:1203] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:00.390827 7206 raft_consensus.cc:2932] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.391355 8803 tablet_copy_client.cc:523] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.392601 8803 tablet_bootstrap.cc:492] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:01:00.392638 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5e922f7cee8046e7b4263dd57b8d1185 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:00.394613 803 raft_consensus.cc:2932] T 5e922f7cee8046e7b4263dd57b8d1185 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.399001 8788 tablet_bootstrap.cc:492] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.399273 8788 tablet_bootstrap.cc:492] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap complete.
I20240502 14:01:00.399467 8788 ts_tablet_manager.cc:1367] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.002s
I20240502 14:01:00.400027 8788 raft_consensus.cc:348] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:01:00.400182 8788 raft_consensus.cc:729] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: LEARNER
I20240502 14:01:00.400190 32731 catalog_manager.cc:5484] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:00.400292 8788 consensus_queue.cc:260] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:01:00.400612 8788 ts_tablet_manager.cc:1398] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:00.402349 2905 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session 452ecbc085e54db49c3b01c20a96f860-79db2b3bbcb041f6be0d86b150a3d324 received from {username='slave'} at 127.26.25.134:33823
I20240502 14:01:00.402432 2905 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session 452ecbc085e54db49c3b01c20a96f860-79db2b3bbcb041f6be0d86b150a3d324 on tablet 79db2b3bbcb041f6be0d86b150a3d324 with peer 452ecbc085e54db49c3b01c20a96f860
I20240502 14:01:00.402954 3018 raft_consensus.cc:2932] T 5e922f7cee8046e7b4263dd57b8d1185 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.404871 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e922f7cee8046e7b4263dd57b8d1185 with delete_type TABLET_DATA_TOMBSTONED (TS da3f8160557e454f8547f45e80c2f013 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47052
I20240502 14:01:00.408354 8815 tablet_replica.cc:330] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:01:00.408519 3152 raft_consensus.cc:1203] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:00.408744 8815 raft_consensus.cc:2229] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:00.409005 8815 raft_consensus.cc:2258] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:00.409837 8815 ts_tablet_manager.cc:1875] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:00.411484 8573 ts_tablet_manager.cc:902] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9: Initiating tablet copy from peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:01:00.411963 8573 tablet_copy_client.cc:308] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.136:43131
I20240502 14:01:00.412530 3304 tablet_copy_service.cc:140] P da3f8160557e454f8547f45e80c2f013: Received BeginTabletCopySession request for tablet be6290bc7df6465f8a43aae431b34f8b from peer 58107b611f8f475a910a6fd54dfd1fb9 ({username='slave'} at 127.26.25.131:54755)
I20240502 14:01:00.412621 3304 tablet_copy_service.cc:161] P da3f8160557e454f8547f45e80c2f013: Beginning new tablet copy session on tablet be6290bc7df6465f8a43aae431b34f8b from peer 58107b611f8f475a910a6fd54dfd1fb9 at {username='slave'} at 127.26.25.131:54755: session id = 58107b611f8f475a910a6fd54dfd1fb9-be6290bc7df6465f8a43aae431b34f8b
I20240502 14:01:00.413504 3304 tablet_copy_source_session.cc:205] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.413738 8815 ts_tablet_manager.cc:1888] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:00.413889 8815 log.cc:1196] T 5e922f7cee8046e7b4263dd57b8d1185 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/5e922f7cee8046e7b4263dd57b8d1185
I20240502 14:01:00.416046 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 5e922f7cee8046e7b4263dd57b8d1185 (table rebalancer_extra_table_0 [id=c9979d0690074e1484d971eed95aad38]) successfully deleted
I20240502 14:01:00.416729 8573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be6290bc7df6465f8a43aae431b34f8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.418694 8573 tablet_copy_client.cc:783] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.418859 8573 tablet_copy_client.cc:647] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.421226 8573 tablet_copy_client.cc:523] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.422693 8573 tablet_bootstrap.cc:492] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:01:00.426920 8573 tablet_bootstrap.cc:492] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.427186 8573 tablet_bootstrap.cc:492] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap complete.
I20240502 14:01:00.427366 8573 ts_tablet_manager.cc:1367] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 14:01:00.427919 8573 raft_consensus.cc:348] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:00.428053 8573 raft_consensus.cc:729] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: LEARNER
I20240502 14:01:00.428165 8573 consensus_queue.cc:260] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:00.428472 8573 ts_tablet_manager.cc:1398] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:01:00.428541 8803 tablet_bootstrap.cc:492] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.428860 8803 tablet_bootstrap.cc:492] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:01:00.429052 8803 ts_tablet_manager.cc:1367] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.036s user 0.022s sys 0.002s
I20240502 14:01:00.429473 8803 raft_consensus.cc:348] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:01:00.429594 8803 raft_consensus.cc:729] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:01:00.429697 8803 consensus_queue.cc:260] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:01:00.431048 8803 ts_tablet_manager.cc:1398] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:00.431511 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-08f30aab835c4380b8b7a01400e5aa55 received from {username='slave'} at 127.26.25.136:47487
I20240502 14:01:00.431586 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session da3f8160557e454f8547f45e80c2f013-08f30aab835c4380b8b7a01400e5aa55 on tablet 08f30aab835c4380b8b7a01400e5aa55 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:01:00.431918 3304 tablet_copy_service.cc:342] P da3f8160557e454f8547f45e80c2f013: Request end of tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-be6290bc7df6465f8a43aae431b34f8b received from {username='slave'} at 127.26.25.131:54755
I20240502 14:01:00.431998 3304 tablet_copy_service.cc:433] P da3f8160557e454f8547f45e80c2f013: ending tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-be6290bc7df6465f8a43aae431b34f8b on tablet be6290bc7df6465f8a43aae431b34f8b with peer 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:01:00.437043 2884 raft_consensus.cc:1203] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:00.463317 3152 consensus_queue.cc:237] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
W20240502 14:01:00.465154 3086 consensus_peers.cc:485] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7450ad74adcc49c3820f93b568ee7c21. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:00.465163 803 raft_consensus.cc:1261] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.465553 6625 consensus_queue.cc:1035] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:00.466176 541 raft_consensus.cc:1261] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.466648 6625 consensus_queue.cc:1035] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:00.468276 8159 raft_consensus.cc:2932] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:01:00.468688 803 raft_consensus.cc:2932] T 7450ad74adcc49c3820f93b568ee7c21 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:01:00.468946 541 raft_consensus.cc:2932] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:01:00.471769 32729 catalog_manager.cc:5484] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:00.478802 8788 ts_tablet_manager.cc:902] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:01:00.479254 8788 tablet_copy_client.cc:308] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:01:00.479784 2905 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 4c27c9fb983c49a59a2f84776775e1bd from peer 452ecbc085e54db49c3b01c20a96f860 ({username='slave'} at 127.26.25.134:33823)
I20240502 14:01:00.479940 2905 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 4c27c9fb983c49a59a2f84776775e1bd from peer 452ecbc085e54db49c3b01c20a96f860 at {username='slave'} at 127.26.25.134:33823: session id = 452ecbc085e54db49c3b01c20a96f860-4c27c9fb983c49a59a2f84776775e1bd
I20240502 14:01:00.480969 2905 tablet_copy_source_session.cc:205] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.481918 8788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c27c9fb983c49a59a2f84776775e1bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.484244 8788 tablet_copy_client.cc:783] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.484405 8788 tablet_copy_client.cc:647] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.485697 8788 tablet_copy_client.cc:523] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.486840 8788 tablet_bootstrap.cc:492] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:01:00.492712 8788 tablet_bootstrap.cc:492] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.492962 8788 tablet_bootstrap.cc:492] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860: Bootstrap complete.
I20240502 14:01:00.493106 8788 ts_tablet_manager.cc:1367] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20240502 14:01:00.493517 8788 raft_consensus.cc:348] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:01:00.493623 8788 raft_consensus.cc:729] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: LEARNER
I20240502 14:01:00.493703 8788 consensus_queue.cc:260] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:01:00.493947 8788 ts_tablet_manager.cc:1398] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:00.496294 2905 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session 452ecbc085e54db49c3b01c20a96f860-4c27c9fb983c49a59a2f84776775e1bd received from {username='slave'} at 127.26.25.134:33823
I20240502 14:01:00.496376 2905 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session 452ecbc085e54db49c3b01c20a96f860-4c27c9fb983c49a59a2f84776775e1bd on tablet 4c27c9fb983c49a59a2f84776775e1bd with peer 452ecbc085e54db49c3b01c20a96f860
I20240502 14:01:00.540179 803 raft_consensus.cc:1203] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:00.583209 7615 raft_consensus.cc:979] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: : Instructing follower 58107b611f8f475a910a6fd54dfd1fb9 to start an election
I20240502 14:01:00.583324 7615 raft_consensus.cc:1067] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Signalling peer 58107b611f8f475a910a6fd54dfd1fb9 to start an election
I20240502 14:01:00.584000 672 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b20d05c7433e4f56832233e2fc9adc84"
dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
from {username='slave'} at 127.26.25.136:47487
I20240502 14:01:00.584163 672 raft_consensus.cc:482] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:00.584226 672 raft_consensus.cc:3037] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:00.585233 672 raft_consensus.cc:504] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:00.585696 672 leader_election.cc:288] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:01:00.586041 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20d05c7433e4f56832233e2fc9adc84" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:01:00.586159 541 raft_consensus.cc:3037] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:00.586230 2884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20d05c7433e4f56832233e2fc9adc84" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:01:00.586372 2884 raft_consensus.cc:3037] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:00.586526 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20d05c7433e4f56832233e2fc9adc84" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:01:00.586629 3283 raft_consensus.cc:3032] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:00.586664 3283 raft_consensus.cc:729] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:01:00.586791 3283 consensus_queue.cc:260] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:00.586957 3283 raft_consensus.cc:3037] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:00.587172 541 raft_consensus.cc:2454] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 2.
I20240502 14:01:00.587414 2884 raft_consensus.cc:2454] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 2.
I20240502 14:01:00.587728 609 leader_election.cc:302] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:01:00.587934 7982 raft_consensus.cc:2781] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:00.588035 7982 raft_consensus.cc:686] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:01:00.588023 3283 raft_consensus.cc:2454] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 2.
I20240502 14:01:00.588187 7982 consensus_queue.cc:237] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:00.589783 3283 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b20d05c7433e4f56832233e2fc9adc84"
dest_uuid: "da3f8160557e454f8547f45e80c2f013"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46356
I20240502 14:01:00.589902 3283 raft_consensus.cc:595] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Received request to transfer leadership
I20240502 14:01:00.589947 3283 raft_consensus.cc:603] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20240502 14:01:00.591018 32729 catalog_manager.cc:5484] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 1 to 2, leader changed from da3f8160557e454f8547f45e80c2f013 (127.26.25.136) to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:00.606873 8939 ts_tablet_manager.cc:902] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:01:00.607344 8939 tablet_copy_client.cc:308] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35251
I20240502 14:01:00.607890 562 tablet_copy_service.cc:140] P c351a89a5fe34e788b9d8064fbffa751: Received BeginTabletCopySession request for tablet c2ce30696a474d23818c3f70cc86e9dd from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:42705)
I20240502 14:01:00.608021 562 tablet_copy_service.cc:161] P c351a89a5fe34e788b9d8064fbffa751: Beginning new tablet copy session on tablet c2ce30696a474d23818c3f70cc86e9dd from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:42705: session id = f003b732dd0f46958262b42f8d20305e-c2ce30696a474d23818c3f70cc86e9dd
I20240502 14:01:00.610100 562 tablet_copy_source_session.cc:205] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.611151 8939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ce30696a474d23818c3f70cc86e9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.613922 8939 tablet_copy_client.cc:783] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.614077 8939 tablet_copy_client.cc:647] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.628945 8939 tablet_copy_client.cc:523] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.631565 8939 tablet_bootstrap.cc:492] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:01:00.635462 8939 tablet_bootstrap.cc:492] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.635845 8939 tablet_bootstrap.cc:492] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:01:00.636124 8939 ts_tablet_manager.cc:1367] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240502 14:01:00.636725 8939 raft_consensus.cc:348] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:01:00.636930 8939 raft_consensus.cc:729] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:01:00.637081 8939 consensus_queue.cc:260] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:01:00.637607 8939 ts_tablet_manager.cc:1398] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:00.639871 562 tablet_copy_service.cc:342] P c351a89a5fe34e788b9d8064fbffa751: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-c2ce30696a474d23818c3f70cc86e9dd received from {username='slave'} at 127.26.25.129:42705
I20240502 14:01:00.639957 562 tablet_copy_service.cc:433] P c351a89a5fe34e788b9d8064fbffa751: ending tablet copy session f003b732dd0f46958262b42f8d20305e-c2ce30696a474d23818c3f70cc86e9dd on tablet c2ce30696a474d23818c3f70cc86e9dd with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:01:00.664698 3150 consensus_queue.cc:237] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:01:00.666260 404 raft_consensus.cc:1261] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.666270 3018 raft_consensus.cc:1261] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:00.666663 8159 consensus_queue.cc:1035] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:00.666852 7246 consensus_queue.cc:1035] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:01:00.667273 3087 consensus_peers.cc:485] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: adaf3fcf84de4223876c7601bdfd45ea. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:00.669023 7246 raft_consensus.cc:2932] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:00.669395 405 raft_consensus.cc:2932] T adaf3fcf84de4223876c7601bdfd45ea P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:00.670226 3019 raft_consensus.cc:2932] T adaf3fcf84de4223876c7601bdfd45ea P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:00.674361 32732 catalog_manager.cc:5484] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:00.731397 8573 ts_tablet_manager.cc:902] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9: Initiating tablet copy from peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:01:00.731926 8573 tablet_copy_client.cc:308] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:44715
I20240502 14:01:00.732584 429 tablet_copy_service.cc:140] P f003b732dd0f46958262b42f8d20305e: Received BeginTabletCopySession request for tablet 604579a2517d4c2eaf855db299d15d5e from peer 58107b611f8f475a910a6fd54dfd1fb9 ({username='slave'} at 127.26.25.131:52191)
I20240502 14:01:00.732690 429 tablet_copy_service.cc:161] P f003b732dd0f46958262b42f8d20305e: Beginning new tablet copy session on tablet 604579a2517d4c2eaf855db299d15d5e from peer 58107b611f8f475a910a6fd54dfd1fb9 at {username='slave'} at 127.26.25.131:52191: session id = 58107b611f8f475a910a6fd54dfd1fb9-604579a2517d4c2eaf855db299d15d5e
I20240502 14:01:00.733733 429 tablet_copy_source_session.cc:205] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.734696 8573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 604579a2517d4c2eaf855db299d15d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.737011 8573 tablet_copy_client.cc:783] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.737170 8573 tablet_copy_client.cc:647] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.739159 8573 tablet_copy_client.cc:523] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.740419 8573 tablet_bootstrap.cc:492] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:01:00.744823 8573 tablet_bootstrap.cc:492] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.745350 8573 tablet_bootstrap.cc:492] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap complete.
I20240502 14:01:00.745515 8573 ts_tablet_manager.cc:1367] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240502 14:01:00.746043 8573 raft_consensus.cc:348] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:00.746176 8573 raft_consensus.cc:729] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: LEARNER
I20240502 14:01:00.746299 8573 consensus_queue.cc:260] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:00.746588 8573 ts_tablet_manager.cc:1398] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:00.748319 429 tablet_copy_service.cc:342] P f003b732dd0f46958262b42f8d20305e: Request end of tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-604579a2517d4c2eaf855db299d15d5e received from {username='slave'} at 127.26.25.131:52191
I20240502 14:01:00.748399 429 tablet_copy_service.cc:433] P f003b732dd0f46958262b42f8d20305e: ending tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-604579a2517d4c2eaf855db299d15d5e on tablet 604579a2517d4c2eaf855db299d15d5e with peer 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:01:00.758656 8573 ts_tablet_manager.cc:902] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9: Initiating tablet copy from peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:01:00.759182 8573 tablet_copy_client.cc:308] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.134:39061
I20240502 14:01:00.759735 3039 tablet_copy_service.cc:140] P 452ecbc085e54db49c3b01c20a96f860: Received BeginTabletCopySession request for tablet 10c70369a07e45afb7bd885b58daec40 from peer 58107b611f8f475a910a6fd54dfd1fb9 ({username='slave'} at 127.26.25.131:44533)
I20240502 14:01:00.759860 3039 tablet_copy_service.cc:161] P 452ecbc085e54db49c3b01c20a96f860: Beginning new tablet copy session on tablet 10c70369a07e45afb7bd885b58daec40 from peer 58107b611f8f475a910a6fd54dfd1fb9 at {username='slave'} at 127.26.25.131:44533: session id = 58107b611f8f475a910a6fd54dfd1fb9-10c70369a07e45afb7bd885b58daec40
I20240502 14:01:00.760787 3039 tablet_copy_source_session.cc:205] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.761705 8573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10c70369a07e45afb7bd885b58daec40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.763880 8573 tablet_copy_client.cc:783] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.764026 8573 tablet_copy_client.cc:647] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.765944 8573 tablet_copy_client.cc:523] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.766979 8573 tablet_bootstrap.cc:492] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:01:00.771389 8573 tablet_bootstrap.cc:492] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.772312 8573 tablet_bootstrap.cc:492] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap complete.
I20240502 14:01:00.772485 8573 ts_tablet_manager.cc:1367] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20240502 14:01:00.773056 8573 raft_consensus.cc:348] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:00.773164 8573 raft_consensus.cc:729] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: LEARNER
I20240502 14:01:00.773267 8573 consensus_queue.cc:260] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:00.773542 8573 ts_tablet_manager.cc:1398] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:00.775187 3039 tablet_copy_service.cc:342] P 452ecbc085e54db49c3b01c20a96f860: Request end of tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-10c70369a07e45afb7bd885b58daec40 received from {username='slave'} at 127.26.25.131:44533
I20240502 14:01:00.775276 3039 tablet_copy_service.cc:433] P 452ecbc085e54db49c3b01c20a96f860: ending tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-10c70369a07e45afb7bd885b58daec40 on tablet 10c70369a07e45afb7bd885b58daec40 with peer 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:01:00.780354 540 raft_consensus.cc:1203] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:00.790544 8803 ts_tablet_manager.cc:902] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:00.790992 8803 tablet_copy_client.cc:308] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.132:33443
I20240502 14:01:00.791596 826 tablet_copy_service.cc:140] P 478dfa53d7944f38b4e6cabee21b2d01: Received BeginTabletCopySession request for tablet 167e1eb3a4034f3ca75e721b8140d757 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:32835)
I20240502 14:01:00.791726 826 tablet_copy_service.cc:161] P 478dfa53d7944f38b4e6cabee21b2d01: Beginning new tablet copy session on tablet 167e1eb3a4034f3ca75e721b8140d757 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:32835: session id = da3f8160557e454f8547f45e80c2f013-167e1eb3a4034f3ca75e721b8140d757
I20240502 14:01:00.793082 826 tablet_copy_source_session.cc:205] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.794560 8803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 167e1eb3a4034f3ca75e721b8140d757. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.800145 3018 raft_consensus.cc:1203] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:00.806655 8803 tablet_copy_client.cc:783] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.806876 8803 tablet_copy_client.cc:647] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.810961 8803 tablet_copy_client.cc:523] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.812426 8803 tablet_bootstrap.cc:492] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:01:00.816264 8803 tablet_bootstrap.cc:492] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.816514 8803 tablet_bootstrap.cc:492] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:01:00.816674 8803 ts_tablet_manager.cc:1367] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20240502 14:01:00.817549 8803 raft_consensus.cc:348] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:01:00.817674 8803 raft_consensus.cc:729] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:01:00.817775 8803 consensus_queue.cc:260] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:01:00.818094 8803 ts_tablet_manager.cc:1398] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:00.823913 826 tablet_copy_service.cc:342] P 478dfa53d7944f38b4e6cabee21b2d01: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-167e1eb3a4034f3ca75e721b8140d757 received from {username='slave'} at 127.26.25.136:32835
I20240502 14:01:00.824014 826 tablet_copy_service.cc:433] P 478dfa53d7944f38b4e6cabee21b2d01: ending tablet copy session da3f8160557e454f8547f45e80c2f013-167e1eb3a4034f3ca75e721b8140d757 on tablet 167e1eb3a4034f3ca75e721b8140d757 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:01:00.826102 9035 ts_tablet_manager.cc:902] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:01:00.829313 9035 tablet_copy_client.cc:308] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:01:00.829865 2905 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet f3d9ae2ed9a84c2c928dc1d03f325fa3 from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:41861)
I20240502 14:01:00.829977 2905 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet f3d9ae2ed9a84c2c928dc1d03f325fa3 from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:41861: session id = 478dfa53d7944f38b4e6cabee21b2d01-f3d9ae2ed9a84c2c928dc1d03f325fa3
I20240502 14:01:00.831012 2905 tablet_copy_source_session.cc:205] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.831990 9035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d9ae2ed9a84c2c928dc1d03f325fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.834029 9035 tablet_copy_client.cc:783] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.834252 9035 tablet_copy_client.cc:647] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.837783 9035 tablet_copy_client.cc:523] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.839387 7206 raft_consensus.cc:1050] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 to VOTER
I20240502 14:01:00.839938 7206 consensus_queue.cc:237] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:00.841567 804 raft_consensus.cc:1261] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.842347 7207 consensus_queue.cc:1035] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.842474 3018 raft_consensus.cc:1261] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.842941 673 raft_consensus.cc:1261] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.844358 9035 tablet_bootstrap.cc:492] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:01:00.844568 7207 consensus_queue.cc:1035] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.844736 8358 consensus_queue.cc:1035] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.850275 9035 tablet_bootstrap.cc:492] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.850934 9035 tablet_bootstrap.cc:492] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:01:00.851157 9035 ts_tablet_manager.cc:1367] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20240502 14:01:00.851969 9035 raft_consensus.cc:348] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:01:00.852124 9035 raft_consensus.cc:729] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:01:00.852252 9035 consensus_queue.cc:260] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:01:00.852602 9035 ts_tablet_manager.cc:1398] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:00.852624 6939 raft_consensus.cc:2932] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:00.854559 3018 raft_consensus.cc:2932] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:00.856637 2905 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-f3d9ae2ed9a84c2c928dc1d03f325fa3 received from {username='slave'} at 127.26.25.132:41861
I20240502 14:01:00.856726 2905 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-f3d9ae2ed9a84c2c928dc1d03f325fa3 on tablet f3d9ae2ed9a84c2c928dc1d03f325fa3 with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:00.856904 804 raft_consensus.cc:2932] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:00.858407 32728 catalog_manager.cc:5484] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:00.860077 673 raft_consensus.cc:2932] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:00.868247 8939 ts_tablet_manager.cc:902] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:01:00.868741 8939 tablet_copy_client.cc:308] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:01:00.869254 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet 7450ad74adcc49c3820f93b568ee7c21 from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:55289)
I20240502 14:01:00.869382 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet 7450ad74adcc49c3820f93b568ee7c21 from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:55289: session id = f003b732dd0f46958262b42f8d20305e-7450ad74adcc49c3820f93b568ee7c21
I20240502 14:01:00.870607 3018 raft_consensus.cc:1203] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:00.870734 3172 tablet_copy_source_session.cc:205] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:00.872097 8939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7450ad74adcc49c3820f93b568ee7c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:00.874229 8939 tablet_copy_client.cc:783] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:00.874379 8939 tablet_copy_client.cc:647] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:00.875172 2885 consensus_queue.cc:237] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:00.877065 673 raft_consensus.cc:1261] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:00.877499 8358 consensus_queue.cc:1035] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:00.878491 804 raft_consensus.cc:1261] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:00.879282 8358 consensus_queue.cc:1035] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:00.882367 8358 raft_consensus.cc:1050] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:01:00.882759 8358 consensus_queue.cc:237] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:01:00.883539 3283 raft_consensus.cc:1203] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:00.885771 8358 raft_consensus.cc:2932] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:00.886191 673 raft_consensus.cc:2932] T 9e642731d7684e33b82eabf97c380123 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:00.887847 804 raft_consensus.cc:1261] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.887881 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9e642731d7684e33b82eabf97c380123 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:00.888590 8939 tablet_copy_client.cc:523] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:00.889371 3284 raft_consensus.cc:1261] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.890446 804 raft_consensus.cc:2932] T 9e642731d7684e33b82eabf97c380123 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:00.890836 6939 consensus_queue.cc:1035] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.892549 6939 consensus_queue.cc:1035] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.893034 32731 catalog_manager.cc:5484] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:00.893266 3150 raft_consensus.cc:1261] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.894014 7207 consensus_queue.cc:1035] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.896293 8939 tablet_bootstrap.cc:492] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:01:00.897881 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e642731d7684e33b82eabf97c380123 with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:01:00.901618 7207 raft_consensus.cc:2932] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:01:00.902194 9062 tablet_replica.cc:330] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:01:00.902149 3150 raft_consensus.cc:2932] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:01:00.902325 9062 raft_consensus.cc:2229] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:00.902472 9062 raft_consensus.cc:2258] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:00.903332 8939 tablet_bootstrap.cc:492] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:00.903604 8939 tablet_bootstrap.cc:492] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:01:00.903869 8939 ts_tablet_manager.cc:1367] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.002s
I20240502 14:01:00.904323 8939 raft_consensus.cc:348] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:01:00.904469 8939 raft_consensus.cc:729] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:01:00.904564 8939 consensus_queue.cc:260] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:01:00.905764 32728 catalog_manager.cc:5484] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:00.906199 8939 ts_tablet_manager.cc:1398] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:00.907039 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-7450ad74adcc49c3820f93b568ee7c21 received from {username='slave'} at 127.26.25.129:55289
I20240502 14:01:00.907122 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session f003b732dd0f46958262b42f8d20305e-7450ad74adcc49c3820f93b568ee7c21 on tablet 7450ad74adcc49c3820f93b568ee7c21 with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:01:00.907184 9062 ts_tablet_manager.cc:1875] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:00.908223 803 raft_consensus.cc:2932] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:01:00.909225 2885 consensus_queue.cc:237] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:01:00.912111 9062 ts_tablet_manager.cc:1888] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:00.912200 9062 log.cc:1196] T 9e642731d7684e33b82eabf97c380123 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/9e642731d7684e33b82eabf97c380123
I20240502 14:01:00.913127 803 raft_consensus.cc:1261] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:00.913956 6939 consensus_queue.cc:1035] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:00.914867 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 9e642731d7684e33b82eabf97c380123 (table rebalancer_extra_table_56 [id=c36d55efbf0d492381cd64ae4e1f17e5]) successfully deleted
I20240502 14:01:00.917025 673 raft_consensus.cc:1203] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:00.917320 6939 raft_consensus.cc:2932] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:01:00.919163 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 00d3e2b37ce743bf90e211f5385efe90 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:00.919971 804 raft_consensus.cc:2932] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:01:00.923506 32732 catalog_manager.cc:5484] T 00d3e2b37ce743bf90e211f5385efe90 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:01:00.924397 3150 raft_consensus.cc:1261] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:00.925689 7207 consensus_queue.cc:1035] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:00.926278 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet 00d3e2b37ce743bf90e211f5385efe90 with delete_type TABLET_DATA_TOMBSTONED (TS da3f8160557e454f8547f45e80c2f013 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47052
I20240502 14:01:00.926729 3150 raft_consensus.cc:2932] T 00d3e2b37ce743bf90e211f5385efe90 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:01:00.928236 9065 tablet_replica.cc:330] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:01:00.928488 9065 raft_consensus.cc:2229] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:00.928658 9065 pending_rounds.cc:62] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013: Trying to abort 1 pending ops.
I20240502 14:01:00.928766 9065 pending_rounds.cc:68] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7023241055774707712 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "00d3e2b37ce743bf90e211f5385efe90" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } } }
I20240502 14:01:00.929329 9065 raft_consensus.cc:2894] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20240502 14:01:00.929472 9065 raft_consensus.cc:2866] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 14:01:00.929584 9065 raft_consensus.cc:2258] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Raft consensus is shut down!
W20240502 14:01:00.930130 9061 consensus_queue.cc:1159] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da3f8160557e454f8547f45e80c2f013" status { last_received { term: 1 index: 3 } last_committed_idx: 2 last_received_current_leader { term: 1 index: 3 } }
I20240502 14:01:00.933706 9065 ts_tablet_manager.cc:1875] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:00.936944 9065 ts_tablet_manager.cc:1888] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:00.937178 9065 log.cc:1196] T 00d3e2b37ce743bf90e211f5385efe90 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/00d3e2b37ce743bf90e211f5385efe90
I20240502 14:01:00.937886 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet 00d3e2b37ce743bf90e211f5385efe90 (table rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]) successfully deleted
I20240502 14:01:00.951620 540 raft_consensus.cc:1203] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:00.956357 5098 raft_consensus.cc:1050] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 to VOTER
I20240502 14:01:00.956837 5098 consensus_queue.cc:237] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:01:00.958366 404 raft_consensus.cc:1261] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:00.958592 2885 raft_consensus.cc:1261] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:00.958818 6065 consensus_queue.cc:1035] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:00.958997 8244 consensus_queue.cc:1035] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:00.959234 540 raft_consensus.cc:1261] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:00.960428 6065 consensus_queue.cc:1035] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:00.962554 8244 raft_consensus.cc:2932] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:00.964457 404 raft_consensus.cc:2932] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:00.967198 540 raft_consensus.cc:2932] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:00.968612 32729 catalog_manager.cc:5484] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:00.968835 5098 raft_consensus.cc:1050] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:01:00.969214 5098 consensus_queue.cc:237] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:01:00.972357 3018 raft_consensus.cc:1261] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.972823 9081 consensus_queue.cc:1035] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.973323 2886 raft_consensus.cc:1261] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.973313 2885 raft_consensus.cc:2932] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:00.974380 804 raft_consensus.cc:1261] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.977389 7982 consensus_queue.cc:1035] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.977641 6065 consensus_queue.cc:1035] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.980680 7615 raft_consensus.cc:1050] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:01:00.981112 7615 consensus_queue.cc:237] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:01:00.984740 3152 raft_consensus.cc:1261] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.985205 8205 consensus_queue.cc:1035] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.985383 404 raft_consensus.cc:1261] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.986675 8205 consensus_queue.cc:1035] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.988536 804 raft_consensus.cc:1261] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:00.988487 673 consensus_queue.cc:237] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:01:00.989078 7615 consensus_queue.cc:1035] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:00.990731 405 raft_consensus.cc:1261] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:00.991106 9081 consensus_queue.cc:1035] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:00.992400 2886 raft_consensus.cc:1261] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:00.992775 5098 consensus_queue.cc:1035] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:00.993386 9091 raft_consensus.cc:2932] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.994032 9081 raft_consensus.cc:2932] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.997045 404 raft_consensus.cc:2932] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.997735 3152 raft_consensus.cc:2932] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.999429 803 raft_consensus.cc:2932] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:00.999774 32728 catalog_manager.cc:5484] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.004381 3018 raft_consensus.cc:2932] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.007952 3283 consensus_queue.cc:237] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:01:01.008523 5098 raft_consensus.cc:2932] T 987eabba58cd472696e64f30db468c71 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:01.009302 32731 catalog_manager.cc:5484] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.010151 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 987eabba58cd472696e64f30db468c71 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.010152 804 raft_consensus.cc:2932] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.010901 2886 raft_consensus.cc:2932] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:01.017796 32726 catalog_manager.cc:5484] T 987eabba58cd472696e64f30db468c71 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:01.018000 3152 raft_consensus.cc:1261] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.020452 409 raft_consensus.cc:2932] T 987eabba58cd472696e64f30db468c71 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:01.021986 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 987eabba58cd472696e64f30db468c71 with delete_type TABLET_DATA_TOMBSTONED (TS c351a89a5fe34e788b9d8064fbffa751 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60890
I20240502 14:01:01.025596 804 raft_consensus.cc:1261] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.028604 2885 raft_consensus.cc:2932] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.031850 409 raft_consensus.cc:1203] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:01.035554 673 consensus_queue.cc:237] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:01:01.036875 3018 raft_consensus.cc:1261] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.037462 6065 consensus_queue.cc:1035] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.038832 804 raft_consensus.cc:1261] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.039474 6065 raft_consensus.cc:2932] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.039938 8244 consensus_queue.cc:1035] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.041479 804 raft_consensus.cc:2932] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.042832 3019 raft_consensus.cc:2932] T 72590739196d48ac88d69b5bcd1124b0 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.022466 9091 consensus_queue.cc:1035] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20240502 14:01:01.049472 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 72590739196d48ac88d69b5bcd1124b0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.049521 8205 consensus_queue.cc:1035] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.050582 32726 catalog_manager.cc:5484] T 72590739196d48ac88d69b5bcd1124b0 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.056669 540 raft_consensus.cc:1261] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:01:01.058693 8244 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.060046 7615 raft_consensus.cc:2932] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.061983 32719 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 81cd2e05295641678fa8d7711c797113 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.063325 3150 raft_consensus.cc:2932] T 81cd2e05295641678fa8d7711c797113 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.063400 803 raft_consensus.cc:2932] T 81cd2e05295641678fa8d7711c797113 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.064301 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 72590739196d48ac88d69b5bcd1124b0 with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:01:01.073277 32728 catalog_manager.cc:5484] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.073740 9035 ts_tablet_manager.cc:902] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:01:01.074270 9035 tablet_copy_client.cc:308] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:01:01.074648 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet adaf3fcf84de4223876c7601bdfd45ea from peer 478dfa53d7944f38b4e6cabee21b2d01 ({username='slave'} at 127.26.25.132:57613)
I20240502 14:01:01.074776 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet adaf3fcf84de4223876c7601bdfd45ea from peer 478dfa53d7944f38b4e6cabee21b2d01 at {username='slave'} at 127.26.25.132:57613: session id = 478dfa53d7944f38b4e6cabee21b2d01-adaf3fcf84de4223876c7601bdfd45ea
I20240502 14:01:01.075908 3172 tablet_copy_source_session.cc:205] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:01.076738 9035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adaf3fcf84de4223876c7601bdfd45ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:01.077186 387 tablet_service.cc:1514] Processing DeleteTablet for tablet 81cd2e05295641678fa8d7711c797113 with delete_type TABLET_DATA_TOMBSTONED (TS f003b732dd0f46958262b42f8d20305e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43314
I20240502 14:01:01.077553 9106 tablet_replica.cc:330] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:01:01.077725 9106 raft_consensus.cc:2229] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.077859 9096 tablet_replica.cc:330] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:01:01.077896 9106 raft_consensus.cc:2258] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.078117 9096 raft_consensus.cc:2229] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.079197 9035 tablet_copy_client.cc:783] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:01.079347 9035 tablet_copy_client.cc:647] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:01.079452 9096 raft_consensus.cc:2258] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.080816 9035 tablet_copy_client.cc:523] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:01.080967 9096 ts_tablet_manager.cc:1875] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.084195 9035 tablet_bootstrap.cc:492] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap starting.
I20240502 14:01:01.087903 9106 ts_tablet_manager.cc:1875] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.089432 9096 ts_tablet_manager.cc:1888] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:01:01.089565 9096 log.cc:1196] T 987eabba58cd472696e64f30db468c71 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/987eabba58cd472696e64f30db468c71
I20240502 14:01:01.090341 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 987eabba58cd472696e64f30db468c71 (table rebalance_test_table_17 [id=147e06cde67f41129125019f57a084e5]) successfully deleted
I20240502 14:01:01.090879 9106 ts_tablet_manager.cc:1888] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:01.090916 9101 tablet_replica.cc:330] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:01:01.090955 9106 log.cc:1196] T 81cd2e05295641678fa8d7711c797113 P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/81cd2e05295641678fa8d7711c797113
I20240502 14:01:01.091065 9101 raft_consensus.cc:2229] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.091220 9101 raft_consensus.cc:2258] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.091468 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 81cd2e05295641678fa8d7711c797113 (table rebalancer_extra_table_74 [id=59ef62e3607c418eb9ab69360784630f]) successfully deleted
I20240502 14:01:01.093261 9101 ts_tablet_manager.cc:1875] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.096333 9035 tablet_bootstrap.cc:492] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:01.096662 9035 tablet_bootstrap.cc:492] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01: Bootstrap complete.
I20240502 14:01:01.096662 9101 ts_tablet_manager.cc:1888] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:01.096843 9035 ts_tablet_manager.cc:1367] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.001s
I20240502 14:01:01.096859 9101 log.cc:1196] T 72590739196d48ac88d69b5bcd1124b0 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/72590739196d48ac88d69b5bcd1124b0
I20240502 14:01:01.097484 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 72590739196d48ac88d69b5bcd1124b0 (table rebalancer_extra_table_72 [id=c4ee0314ba8244b5aa832bb31e2b0534]) successfully deleted
I20240502 14:01:01.097537 9035 raft_consensus.cc:348] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:01:01.097666 9035 raft_consensus.cc:729] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 478dfa53d7944f38b4e6cabee21b2d01, State: Initialized, Role: LEARNER
I20240502 14:01:01.097771 9035 consensus_queue.cc:260] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:01:01.098986 9035 ts_tablet_manager.cc:1398] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:01.099444 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-adaf3fcf84de4223876c7601bdfd45ea received from {username='slave'} at 127.26.25.132:57613
I20240502 14:01:01.099525 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session 478dfa53d7944f38b4e6cabee21b2d01-adaf3fcf84de4223876c7601bdfd45ea on tablet adaf3fcf84de4223876c7601bdfd45ea with peer 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:01.122563 673 raft_consensus.cc:1203] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:01.140061 541 consensus_queue.cc:237] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:01:01.141547 804 raft_consensus.cc:1261] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.141943 672 raft_consensus.cc:1261] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.142313 7384 consensus_queue.cc:1035] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:01:01.142477 477 consensus_peers.cc:485] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): Couldn't send request to peer 4bcd7712aa844d98977a7d7e605c31f5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 556b3a4cab9245d4939ba4c10bf9d4b4. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:01.142576 8662 consensus_queue.cc:1035] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.144385 8662 raft_consensus.cc:2932] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:01.144656 672 raft_consensus.cc:2932] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:01.147857 804 raft_consensus.cc:2932] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:01.148639 32731 catalog_manager.cc:5484] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:01.159590 3284 consensus_queue.cc:237] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:01:01.161085 541 raft_consensus.cc:1261] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:01.161129 3150 raft_consensus.cc:1261] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:01.161432 8205 consensus_queue.cc:1035] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.161599 9091 consensus_queue.cc:1035] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:01:01.161828 3219 consensus_peers.cc:485] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c80435eecda64ecea7864522f5a1dd4c. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:01.163478 7615 raft_consensus.cc:2932] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:01.163975 541 raft_consensus.cc:2932] T c80435eecda64ecea7864522f5a1dd4c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:01.163967 3150 raft_consensus.cc:2932] T c80435eecda64ecea7864522f5a1dd4c P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:01.167999 32731 catalog_manager.cc:5484] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:01.176613 672 consensus_queue.cc:237] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
W20240502 14:01:01.178377 609 consensus_peers.cc:485] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): Couldn't send request to peer 4bcd7712aa844d98977a7d7e605c31f5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f47f72fcaaae46d68f4cdab7595095ca. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:01.178615 804 raft_consensus.cc:1261] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.178622 541 raft_consensus.cc:1261] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.178949 8244 consensus_queue.cc:1035] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.179134 5098 consensus_queue.cc:1035] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.181177 9081 raft_consensus.cc:2932] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:01.181646 541 raft_consensus.cc:2932] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:01.181704 803 raft_consensus.cc:2932] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:01.185101 32731 catalog_manager.cc:5484] T f47f72fcaaae46d68f4cdab7595095ca P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:01.189600 3284 raft_consensus.cc:1261] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:01:01.189973 5098 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.193399 2885 raft_consensus.cc:1261] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:01:01.193841 7982 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.199473 672 raft_consensus.cc:1203] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:01.204430 2885 consensus_queue.cc:237] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
W20240502 14:01:01.205876 2822 consensus_peers.cc:485] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d493cec70a39401b98c21446f866789a. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:01.206167 3284 raft_consensus.cc:1261] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:01.206533 7207 consensus_queue.cc:1035] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.207228 3017 raft_consensus.cc:1261] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:01.207633 7207 consensus_queue.cc:1035] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.208293 8358 raft_consensus.cc:2932] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:01:01.208850 3284 raft_consensus.cc:2932] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:01:01.209785 3017 raft_consensus.cc:2932] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:01:01.212280 32731 catalog_manager.cc:5484] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:01.221796 3284 raft_consensus.cc:1203] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:01.225800 3150 consensus_queue.cc:237] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } }
I20240502 14:01:01.227053 3284 raft_consensus.cc:1261] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:01.227066 409 raft_consensus.cc:1261] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:01:01.227315 3087 consensus_peers.cc:485] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 20937116074f456ea732396f80bd7d6a. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:01.227488 8966 consensus_queue.cc:1035] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.227876 8966 consensus_queue.cc:1035] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.229295 7246 raft_consensus.cc:2932] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:01.229766 409 raft_consensus.cc:2932] T 20937116074f456ea732396f80bd7d6a P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:01.233875 3284 raft_consensus.cc:2932] T 20937116074f456ea732396f80bd7d6a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } } }
I20240502 14:01:01.233983 32728 catalog_manager.cc:5484] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: NON_VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:01.239689 7207 raft_consensus.cc:1050] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER 452ecbc085e54db49c3b01c20a96f860 to VOTER
I20240502 14:01:01.240166 7207 consensus_queue.cc:237] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:01:01.241778 3150 raft_consensus.cc:1261] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.241822 3017 raft_consensus.cc:1261] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.242120 8167 consensus_queue.cc:1035] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.242491 8167 consensus_queue.cc:1035] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.242543 672 raft_consensus.cc:1261] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.243016 8167 consensus_queue.cc:1035] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.245538 803 raft_consensus.cc:1203] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:01.247184 7207 raft_consensus.cc:2932] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.250097 672 raft_consensus.cc:2932] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.250438 3152 raft_consensus.cc:2932] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.253149 3017 raft_consensus.cc:2932] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.256470 32729 catalog_manager.cc:5484] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.262342 2885 consensus_queue.cc:237] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:01:01.263607 3017 raft_consensus.cc:1261] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.263856 672 raft_consensus.cc:1261] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.264142 8167 consensus_queue.cc:1035] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.264344 8358 consensus_queue.cc:1035] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.266495 8358 raft_consensus.cc:2932] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.268628 3017 raft_consensus.cc:2932] T 79db2b3bbcb041f6be0d86b150a3d324 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.268884 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 79db2b3bbcb041f6be0d86b150a3d324 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.269762 32728 catalog_manager.cc:5484] T 79db2b3bbcb041f6be0d86b150a3d324 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.271395 6174 raft_consensus.cc:1050] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01: attempting to promote NON_VOTER c351a89a5fe34e788b9d8064fbffa751 to VOTER
I20240502 14:01:01.272771 6174 consensus_queue.cc:237] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:01.273943 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 79db2b3bbcb041f6be0d86b150a3d324 with delete_type TABLET_DATA_TOMBSTONED (TS 2b6796cf0e2e496e9dd641beb432f485 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60258
I20240502 14:01:01.274507 405 raft_consensus.cc:1261] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.274734 541 raft_consensus.cc:1261] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.274713 672 raft_consensus.cc:2932] T 79db2b3bbcb041f6be0d86b150a3d324 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.275023 3284 raft_consensus.cc:1261] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.275069 8525 consensus_queue.cc:1035] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.275370 8525 consensus_queue.cc:1035] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.277144 8525 consensus_queue.cc:1035] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.279124 9194 tablet_replica.cc:330] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:01:01.279371 9194 raft_consensus.cc:2229] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.279621 9194 raft_consensus.cc:2258] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.280411 9194 ts_tablet_manager.cc:1875] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.281745 9014 raft_consensus.cc:2932] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.283161 9194 ts_tablet_manager.cc:1888] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:01.283288 9194 log.cc:1196] T 79db2b3bbcb041f6be0d86b150a3d324 P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/79db2b3bbcb041f6be0d86b150a3d324
I20240502 14:01:01.284250 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 79db2b3bbcb041f6be0d86b150a3d324 (table rebalancer_extra_table_30 [id=051523fc8cdc4372b1911d17412f7678]) successfully deleted
I20240502 14:01:01.284171 541 raft_consensus.cc:2932] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.284917 7206 raft_consensus.cc:1050] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER 452ecbc085e54db49c3b01c20a96f860 to VOTER
I20240502 14:01:01.285274 7206 consensus_queue.cc:237] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:01:01.286592 3284 raft_consensus.cc:1261] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.286836 32726 catalog_manager.cc:5484] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.287047 7206 consensus_queue.cc:1035] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.287530 3152 raft_consensus.cc:1261] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.287925 3017 raft_consensus.cc:1261] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.287998 7206 consensus_queue.cc:1035] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.288925 3283 raft_consensus.cc:2932] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.289108 7206 consensus_queue.cc:1035] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.291508 405 raft_consensus.cc:2932] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.294268 8358 raft_consensus.cc:2932] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.294785 3017 raft_consensus.cc:2932] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.296168 803 consensus_queue.cc:237] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:01.297279 3152 raft_consensus.cc:2932] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.298101 3284 raft_consensus.cc:2932] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.298305 3283 raft_consensus.cc:1261] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.299117 32726 catalog_manager.cc:5484] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 2 to 3, 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.299955 6174 consensus_queue.cc:1035] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.301558 541 raft_consensus.cc:1261] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.302035 6174 consensus_queue.cc:1035] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.302978 2885 consensus_queue.cc:237] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:01:01.306354 3017 raft_consensus.cc:1261] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.306550 6174 raft_consensus.cc:2932] T 9185ea12db4b413a8c76800b828836da P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.307018 541 raft_consensus.cc:2932] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.307966 8167 consensus_queue.cc:1035] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.308362 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9185ea12db4b413a8c76800b828836da with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.308871 3281 raft_consensus.cc:1261] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.309355 8167 consensus_queue.cc:1035] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.311416 32731 catalog_manager.cc:5484] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.314391 7207 raft_consensus.cc:2932] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.314786 3281 raft_consensus.cc:2932] T 4c27c9fb983c49a59a2f84776775e1bd P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.316061 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4c27c9fb983c49a59a2f84776775e1bd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.318048 32728 catalog_manager.cc:5484] T 4c27c9fb983c49a59a2f84776775e1bd P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:01.319994 3283 raft_consensus.cc:2932] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.320363 387 tablet_service.cc:1514] Processing DeleteTablet for tablet 9185ea12db4b413a8c76800b828836da with delete_type TABLET_DATA_TOMBSTONED (TS f003b732dd0f46958262b42f8d20305e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43314
I20240502 14:01:01.320771 9106 tablet_replica.cc:330] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e: stopping tablet replica
I20240502 14:01:01.320889 9106 raft_consensus.cc:2229] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.320987 9106 raft_consensus.cc:2258] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.321501 3019 raft_consensus.cc:2932] T 4c27c9fb983c49a59a2f84776775e1bd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } } }
I20240502 14:01:01.324180 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c27c9fb983c49a59a2f84776775e1bd with delete_type TABLET_DATA_TOMBSTONED (TS 2b6796cf0e2e496e9dd641beb432f485 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60258
I20240502 14:01:01.325579 9194 tablet_replica.cc:330] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:01:01.326010 9194 raft_consensus.cc:2229] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.326129 9106 ts_tablet_manager.cc:1875] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.326177 9194 raft_consensus.cc:2258] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.328508 9106 ts_tablet_manager.cc:1888] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:01.328591 9106 log.cc:1196] T 9185ea12db4b413a8c76800b828836da P f003b732dd0f46958262b42f8d20305e: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/wals/9185ea12db4b413a8c76800b828836da
I20240502 14:01:01.328590 9194 ts_tablet_manager.cc:1875] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.329176 32720 catalog_manager.cc:4830] TS f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): tablet 9185ea12db4b413a8c76800b828836da (table rebalancer_extra_table_8 [id=72712ccf79f7480baf5c42d1e148d845]) successfully deleted
I20240502 14:01:01.330475 9194 ts_tablet_manager.cc:1888] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:01.330556 9194 log.cc:1196] T 4c27c9fb983c49a59a2f84776775e1bd P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/4c27c9fb983c49a59a2f84776775e1bd
I20240502 14:01:01.331108 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 4c27c9fb983c49a59a2f84776775e1bd (table rebalancer_extra_table_8 [id=72712ccf79f7480baf5c42d1e148d845]) successfully deleted
I20240502 14:01:01.386740 8167 raft_consensus.cc:1050] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER c351a89a5fe34e788b9d8064fbffa751 to VOTER
I20240502 14:01:01.387217 8167 consensus_queue.cc:237] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:01.388656 3017 raft_consensus.cc:1261] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.388758 541 raft_consensus.cc:1261] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.388942 405 raft_consensus.cc:1261] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.389192 8167 consensus_queue.cc:1035] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.389386 7206 consensus_queue.cc:1035] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.390027 7206 consensus_queue.cc:1035] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.393806 8358 raft_consensus.cc:2932] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.395897 541 raft_consensus.cc:2932] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.395980 405 raft_consensus.cc:2932] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.398079 3017 raft_consensus.cc:2932] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.399230 32726 catalog_manager.cc:5484] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.405531 2885 consensus_queue.cc:237] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:01.406659 541 raft_consensus.cc:1261] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.406885 405 raft_consensus.cc:1261] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.407033 6939 consensus_queue.cc:1035] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.407225 8358 consensus_queue.cc:1035] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.409135 8167 raft_consensus.cc:2932] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.409633 541 raft_consensus.cc:2932] T ec443abbb4954bf3b28336e0b5bc0c4b P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.411343 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ec443abbb4954bf3b28336e0b5bc0c4b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.412369 405 raft_consensus.cc:2932] T ec443abbb4954bf3b28336e0b5bc0c4b P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:01.414218 32731 catalog_manager.cc:5484] T ec443abbb4954bf3b28336e0b5bc0c4b P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.419229 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet ec443abbb4954bf3b28336e0b5bc0c4b with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:01:01.419595 9278 tablet_replica.cc:330] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:01:01.419752 9278 raft_consensus.cc:2229] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.419998 9278 raft_consensus.cc:2258] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.420907 9278 ts_tablet_manager.cc:1875] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.423519 9278 ts_tablet_manager.cc:1888] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:01.423645 9278 log.cc:1196] T ec443abbb4954bf3b28336e0b5bc0c4b P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/ec443abbb4954bf3b28336e0b5bc0c4b
I20240502 14:01:01.424300 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet ec443abbb4954bf3b28336e0b5bc0c4b (table rebalancer_extra_table_20 [id=8431fabb3f4e41f39f484115f7bf56b4]) successfully deleted
I20240502 14:01:01.443953 3152 consensus_queue.cc:237] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:01:01.445381 405 raft_consensus.cc:1261] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:01.445470 3283 raft_consensus.cc:1261] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:01:01.445791 3084 consensus_peers.cc:485] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): Couldn't send request to peer 452ecbc085e54db49c3b01c20a96f860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 527784cdca2f4840aa2f2a373a2c6db6. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:01.445861 8159 consensus_queue.cc:1035] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.446027 8966 consensus_queue.cc:1035] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.447731 7246 raft_consensus.cc:2932] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:01:01.448221 405 raft_consensus.cc:2932] T 527784cdca2f4840aa2f2a373a2c6db6 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:01:01.448320 3283 raft_consensus.cc:2932] T 527784cdca2f4840aa2f2a373a2c6db6 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:01:01.451992 32728 catalog_manager.cc:5484] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:01.463840 2885 consensus_queue.cc:237] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:01:01.465685 3284 raft_consensus.cc:1261] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:01.465773 3017 raft_consensus.cc:1261] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:01:01.465936 2819 consensus_peers.cc:485] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0dc1880244a64b9f96ad5b8122a49fce. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:01.466238 7207 consensus_queue.cc:1035] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.466459 8358 consensus_queue.cc:1035] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.467969 405 raft_consensus.cc:1203] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:01.468592 7207 raft_consensus.cc:2932] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:01:01.470289 3284 raft_consensus.cc:2932] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:01:01.472473 3017 raft_consensus.cc:2932] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:01:01.475039 32731 catalog_manager.cc:5484] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index -1 to 2, NON_VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) added. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } } }
I20240502 14:01:01.483952 9081 raft_consensus.cc:1050] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER da3f8160557e454f8547f45e80c2f013 to VOTER
I20240502 14:01:01.484416 9081 consensus_queue.cc:237] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:01:01.485847 405 raft_consensus.cc:1261] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:01.485980 804 raft_consensus.cc:1261] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:01.486616 672 consensus_queue.cc:237] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:01:01.486827 9081 consensus_queue.cc:1035] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:01.487474 8244 consensus_queue.cc:1035] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:01.487521 3284 raft_consensus.cc:1261] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:01.488332 7982 consensus_queue.cc:1035] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:01.489809 2885 raft_consensus.cc:1261] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:01.490059 3284 raft_consensus.cc:1261] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:01:01.490077 609 consensus_peers.cc:485] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c8e843cd7c984ebfb532fec67f97e249. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:01.490327 6065 consensus_queue.cc:1035] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.490480 7982 consensus_queue.cc:1035] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.492342 9081 raft_consensus.cc:2932] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:01:01.492390 6065 raft_consensus.cc:2932] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:01:01.494566 405 raft_consensus.cc:2932] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:01:01.496444 32732 catalog_manager.cc:5484] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:01.496789 32732 catalog_manager.cc:5484] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.497280 3283 raft_consensus.cc:2932] T c8e843cd7c984ebfb532fec67f97e249 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:01:01.497773 9130 raft_consensus.cc:1050] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:01:01.498188 9130 consensus_queue.cc:237] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:01:01.498824 3284 raft_consensus.cc:2932] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:01:01.500591 672 consensus_queue.cc:237] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:01:01.500774 3150 raft_consensus.cc:1261] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.501338 7384 consensus_queue.cc:1035] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.501911 405 raft_consensus.cc:1261] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:01.502310 5098 consensus_queue.cc:1035] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:01.502471 3017 raft_consensus.cc:1261] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.504354 3284 raft_consensus.cc:1261] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:01.505470 2886 raft_consensus.cc:2932] T c8e843cd7c984ebfb532fec67f97e249 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:01:01.505992 8662 consensus_queue.cc:1035] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.509042 8244 consensus_queue.cc:1035] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:01.509896 8662 raft_consensus.cc:2932] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:01.510109 405 raft_consensus.cc:1261] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.510464 8244 raft_consensus.cc:2932] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:01:01.511909 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 08f30aab835c4380b8b7a01400e5aa55 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.512020 3150 raft_consensus.cc:2932] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:01.512964 405 raft_consensus.cc:2932] T 08f30aab835c4380b8b7a01400e5aa55 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:01:01.515719 7384 consensus_queue.cc:1035] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.516283 32731 catalog_manager.cc:5484] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:01.517084 32726 catalog_manager.cc:5484] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:01.517630 3019 raft_consensus.cc:2932] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:01.521667 785 tablet_service.cc:1514] Processing DeleteTablet for tablet 08f30aab835c4380b8b7a01400e5aa55 with delete_type TABLET_DATA_TOMBSTONED (TS 478dfa53d7944f38b4e6cabee21b2d01 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:39484
I20240502 14:01:01.523344 405 raft_consensus.cc:2932] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:01.525723 9328 tablet_replica.cc:330] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01: stopping tablet replica
I20240502 14:01:01.525933 9328 raft_consensus.cc:2229] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.526495 9328 pending_rounds.cc:62] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01: Trying to abort 1 pending ops.
I20240502 14:01:01.527789 9328 pending_rounds.cc:68] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01: Aborting op as it isn't in flight: id { term: 1 index: 5 } timestamp: 7023241058238967808 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "08f30aab835c4380b8b7a01400e5aa55" old_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } } new_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } } }
I20240502 14:01:01.528203 9328 raft_consensus.cc:2894] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Aborting config change with OpId 1.5: Aborted: Op aborted
I20240502 14:01:01.528334 9328 raft_consensus.cc:2866] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 14:01:01.528604 9328 raft_consensus.cc:2258] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.529821 9328 ts_tablet_manager.cc:1875] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.531857 3284 raft_consensus.cc:2932] T 08f30aab835c4380b8b7a01400e5aa55 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:01:01.533486 9091 raft_consensus.cc:1050] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013: attempting to promote NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 to VOTER
I20240502 14:01:01.533907 9091 consensus_queue.cc:237] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:01.535270 672 raft_consensus.cc:1261] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.535744 9091 consensus_queue.cc:1035] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.536079 3017 raft_consensus.cc:1261] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.536873 2886 raft_consensus.cc:1261] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.537356 8205 consensus_queue.cc:1035] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.537655 8205 consensus_queue.cc:1035] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.539688 8205 raft_consensus.cc:2932] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.540455 3017 raft_consensus.cc:2932] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.541884 672 consensus_queue.cc:237] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:01:01.543119 9328 ts_tablet_manager.cc:1888] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:01:01.543323 541 raft_consensus.cc:1261] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.543365 804 raft_consensus.cc:1261] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.543196 9328 log.cc:1196] T 08f30aab835c4380b8b7a01400e5aa55 P 478dfa53d7944f38b4e6cabee21b2d01: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/wals/08f30aab835c4380b8b7a01400e5aa55
I20240502 14:01:01.544210 32721 catalog_manager.cc:4830] TS 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): tablet 08f30aab835c4380b8b7a01400e5aa55 (table rebalance_test_table_7 [id=3dec3eec8fda4142913bf2d106d579b1]) successfully deleted
I20240502 14:01:01.544339 2886 raft_consensus.cc:2932] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
W20240502 14:01:01.544591 609 consensus_peers.cc:485] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): Couldn't send request to peer 4bcd7712aa844d98977a7d7e605c31f5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8689975c1d9d41f4ae7152bf6e4a58a8. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:01.544924 32726 catalog_manager.cc:5484] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.546921 6065 consensus_queue.cc:1035] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.547266 6065 consensus_queue.cc:1035] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.548090 3283 consensus_queue.cc:237] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:01.549345 672 raft_consensus.cc:2932] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.550940 3017 raft_consensus.cc:1261] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.551589 8205 consensus_queue.cc:1035] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.551672 672 raft_consensus.cc:1261] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer da3f8160557e454f8547f45e80c2f013: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.551959 6065 raft_consensus.cc:2932] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:01.552065 9091 consensus_queue.cc:1035] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.553680 9091 raft_consensus.cc:2932] T be6290bc7df6465f8a43aae431b34f8b P da3f8160557e454f8547f45e80c2f013 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.553799 804 raft_consensus.cc:2932] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:01.555526 32719 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet be6290bc7df6465f8a43aae431b34f8b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.555528 3017 raft_consensus.cc:2932] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.557788 32729 catalog_manager.cc:5484] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:01.558743 32726 catalog_manager.cc:5484] T be6290bc7df6465f8a43aae431b34f8b P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.560005 541 raft_consensus.cc:2932] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:01.560022 672 raft_consensus.cc:2932] T be6290bc7df6465f8a43aae431b34f8b P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.564858 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet be6290bc7df6465f8a43aae431b34f8b with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:01:01.565081 9101 tablet_replica.cc:330] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:01:01.565217 9101 raft_consensus.cc:2229] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.565335 9101 raft_consensus.cc:2258] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.566259 9101 ts_tablet_manager.cc:1875] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.568238 9101 ts_tablet_manager.cc:1888] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:01.568311 9101 log.cc:1196] T be6290bc7df6465f8a43aae431b34f8b P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/be6290bc7df6465f8a43aae431b34f8b
I20240502 14:01:01.568835 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet be6290bc7df6465f8a43aae431b34f8b (table rebalancer_extra_table_26 [id=a73bce19098e4a239748caf030e927aa]) successfully deleted
I20240502 14:01:01.573683 9343 ts_tablet_manager.cc:902] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:01:01.574311 9343 tablet_copy_client.cc:308] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:01:01.574554 3150 consensus_queue.cc:237] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:01:01.575729 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet f47f72fcaaae46d68f4cdab7595095ca from peer 4bcd7712aa844d98977a7d7e605c31f5 ({username='slave'} at 127.26.25.133:46031)
I20240502 14:01:01.575904 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet f47f72fcaaae46d68f4cdab7595095ca from peer 4bcd7712aa844d98977a7d7e605c31f5 at {username='slave'} at 127.26.25.133:46031: session id = 4bcd7712aa844d98977a7d7e605c31f5-f47f72fcaaae46d68f4cdab7595095ca
I20240502 14:01:01.576427 541 raft_consensus.cc:1261] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:01.576543 404 raft_consensus.cc:1261] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:01:01.576761 3084 consensus_peers.cc:485] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): Couldn't send request to peer 452ecbc085e54db49c3b01c20a96f860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9bf2e05fba1943edb5cd00989f93df7c. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:01.576848 693 tablet_copy_source_session.cc:205] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:01.576951 8966 consensus_queue.cc:1035] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.577112 8159 consensus_queue.cc:1035] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:01.577762 9343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f47f72fcaaae46d68f4cdab7595095ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:01.578979 8966 raft_consensus.cc:2932] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:01:01.579973 404 raft_consensus.cc:2932] T 9bf2e05fba1943edb5cd00989f93df7c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:01:01.580394 9343 tablet_copy_client.cc:783] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:01.580531 9343 tablet_copy_client.cc:647] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:01.580818 541 raft_consensus.cc:2932] T 9bf2e05fba1943edb5cd00989f93df7c P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } } }
I20240502 14:01:01.584077 32729 catalog_manager.cc:5484] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:01.592341 672 consensus_queue.cc:237] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:01.592978 9343 tablet_copy_client.cc:523] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:01.593940 2886 raft_consensus.cc:1261] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:01.594308 7982 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:01.594659 541 raft_consensus.cc:1261] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:01.594861 9343 tablet_bootstrap.cc:492] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:01:01.595860 7982 consensus_queue.cc:1035] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:01.597074 7982 raft_consensus.cc:2932] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.598749 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b20d05c7433e4f56832233e2fc9adc84 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.598837 541 raft_consensus.cc:2932] T b20d05c7433e4f56832233e2fc9adc84 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.599969 2886 raft_consensus.cc:2932] T b20d05c7433e4f56832233e2fc9adc84 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.602244 32732 catalog_manager.cc:5484] T b20d05c7433e4f56832233e2fc9adc84 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 3 to 5, VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) evicted. New cstate: current_term: 2 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.605584 3263 tablet_service.cc:1514] Processing DeleteTablet for tablet b20d05c7433e4f56832233e2fc9adc84 with delete_type TABLET_DATA_TOMBSTONED (TS da3f8160557e454f8547f45e80c2f013 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47052
I20240502 14:01:01.605963 9357 tablet_replica.cc:330] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: stopping tablet replica
I20240502 14:01:01.606268 9357 raft_consensus.cc:2229] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.606845 9357 raft_consensus.cc:2258] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.607502 9357 ts_tablet_manager.cc:1875] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.616629 9358 ts_tablet_manager.cc:902] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5: Initiating tablet copy from peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:01:01.617358 9358 tablet_copy_client.cc:308] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.130:35251
I20240502 14:01:01.617794 562 tablet_copy_service.cc:140] P c351a89a5fe34e788b9d8064fbffa751: Received BeginTabletCopySession request for tablet 556b3a4cab9245d4939ba4c10bf9d4b4 from peer 4bcd7712aa844d98977a7d7e605c31f5 ({username='slave'} at 127.26.25.133:51585)
I20240502 14:01:01.617889 562 tablet_copy_service.cc:161] P c351a89a5fe34e788b9d8064fbffa751: Beginning new tablet copy session on tablet 556b3a4cab9245d4939ba4c10bf9d4b4 from peer 4bcd7712aa844d98977a7d7e605c31f5 at {username='slave'} at 127.26.25.133:51585: session id = 4bcd7712aa844d98977a7d7e605c31f5-556b3a4cab9245d4939ba4c10bf9d4b4
I20240502 14:01:01.618832 562 tablet_copy_source_session.cc:205] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:01.619968 9358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556b3a4cab9245d4939ba4c10bf9d4b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:01.620071 804 raft_consensus.cc:1203] T adaf3fcf84de4223876c7601bdfd45ea P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:01.626757 9357 ts_tablet_manager.cc:1888] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20240502 14:01:01.626922 9357 log.cc:1196] T b20d05c7433e4f56832233e2fc9adc84 P da3f8160557e454f8547f45e80c2f013: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/wals/b20d05c7433e4f56832233e2fc9adc84
I20240502 14:01:01.627481 9358 tablet_copy_client.cc:783] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:01.627635 9358 tablet_copy_client.cc:647] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:01.628662 9358 tablet_copy_client.cc:523] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:01.629539 8167 raft_consensus.cc:1050] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER 478dfa53d7944f38b4e6cabee21b2d01 to VOTER
I20240502 14:01:01.629887 9358 tablet_bootstrap.cc:492] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:01:01.629961 8167 consensus_queue.cc:237] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:01:01.631484 3017 raft_consensus.cc:1261] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.632091 7207 consensus_queue.cc:1035] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.633126 3150 raft_consensus.cc:1261] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.633854 804 raft_consensus.cc:1261] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.634624 8358 consensus_queue.cc:1035] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.634964 8358 consensus_queue.cc:1035] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.639953 7207 raft_consensus.cc:2932] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.640486 32719 catalog_manager.cc:4830] TS da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): tablet b20d05c7433e4f56832233e2fc9adc84 (table rebalancer_extra_table_40 [id=028df27ba61c4a6ab177037f56a9d843]) successfully deleted
I20240502 14:01:01.641762 3017 raft_consensus.cc:2932] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.641899 804 raft_consensus.cc:2932] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.644763 9343 tablet_bootstrap.cc:492] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:01.644994 9343 tablet_bootstrap.cc:492] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap complete.
I20240502 14:01:01.645114 9343 ts_tablet_manager.cc:1367] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.050s user 0.026s sys 0.000s
I20240502 14:01:01.645124 32732 catalog_manager.cc:5484] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.645524 9343 raft_consensus.cc:348] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:01:01.645608 9343 raft_consensus.cc:729] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: LEARNER
I20240502 14:01:01.645700 9343 consensus_queue.cc:260] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:01:01.646086 9343 ts_tablet_manager.cc:1398] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:01.647533 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-f47f72fcaaae46d68f4cdab7595095ca received from {username='slave'} at 127.26.25.133:46031
I20240502 14:01:01.647622 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-f47f72fcaaae46d68f4cdab7595095ca on tablet f47f72fcaaae46d68f4cdab7595095ca with peer 4bcd7712aa844d98977a7d7e605c31f5
I20240502 14:01:01.657338 8335 raft_consensus.cc:1050] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e: attempting to promote NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 to VOTER
I20240502 14:01:01.657639 3150 raft_consensus.cc:2932] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.657799 8335 consensus_queue.cc:237] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:01.659109 3152 raft_consensus.cc:1261] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.659523 8335 consensus_queue.cc:1035] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.660645 3017 raft_consensus.cc:1261] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.661731 672 raft_consensus.cc:1261] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.664238 8335 consensus_queue.cc:1035] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.664575 8335 consensus_queue.cc:1035] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.664702 2886 consensus_queue.cc:237] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } }
I20240502 14:01:01.666162 3150 raft_consensus.cc:1261] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.666184 804 raft_consensus.cc:1261] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.666603 6939 consensus_queue.cc:1035] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.667685 6939 consensus_queue.cc:1035] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.669651 8167 raft_consensus.cc:2932] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.670064 803 raft_consensus.cc:2932] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.672678 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f3d9ae2ed9a84c2c928dc1d03f325fa3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.674716 9358 tablet_bootstrap.cc:492] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:01.675052 9358 tablet_bootstrap.cc:492] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap complete.
I20240502 14:01:01.675237 9358 ts_tablet_manager.cc:1367] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.045s user 0.022s sys 0.004s
I20240502 14:01:01.675313 8335 raft_consensus.cc:2932] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.675668 9358 raft_consensus.cc:348] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:01:01.675766 9358 raft_consensus.cc:729] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: LEARNER
I20240502 14:01:01.675868 9358 consensus_queue.cc:260] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:01:01.676213 9358 ts_tablet_manager.cc:1398] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:01.676523 32729 catalog_manager.cc:5484] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.677282 562 tablet_copy_service.cc:342] P c351a89a5fe34e788b9d8064fbffa751: Request end of tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-556b3a4cab9245d4939ba4c10bf9d4b4 received from {username='slave'} at 127.26.25.133:51585
I20240502 14:01:01.677362 562 tablet_copy_service.cc:433] P c351a89a5fe34e788b9d8064fbffa751: ending tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-556b3a4cab9245d4939ba4c10bf9d4b4 on tablet 556b3a4cab9245d4939ba4c10bf9d4b4 with peer 4bcd7712aa844d98977a7d7e605c31f5
I20240502 14:01:01.677857 3150 raft_consensus.cc:2932] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.680126 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet f3d9ae2ed9a84c2c928dc1d03f325fa3 with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:01:01.680356 9278 tablet_replica.cc:330] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:01:01.680501 9278 raft_consensus.cc:2229] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.680619 9278 raft_consensus.cc:2258] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.681751 9278 ts_tablet_manager.cc:1875] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.683243 672 raft_consensus.cc:2932] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.683406 32732 catalog_manager.cc:5484] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:01.692395 9278 ts_tablet_manager.cc:1888] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:01.692482 9278 log.cc:1196] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/f3d9ae2ed9a84c2c928dc1d03f325fa3
I20240502 14:01:01.693426 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet f3d9ae2ed9a84c2c928dc1d03f325fa3 (table rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]) successfully deleted
I20240502 14:01:01.696244 9160 raft_consensus.cc:1050] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860: attempting to promote NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 to VOTER
I20240502 14:01:01.696673 9160 consensus_queue.cc:237] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:01.698359 803 raft_consensus.cc:1261] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.698388 2886 raft_consensus.cc:1261] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.698865 8275 consensus_queue.cc:1035] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.699191 8275 consensus_queue.cc:1035] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.699271 672 raft_consensus.cc:1261] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.699190 3152 raft_consensus.cc:2932] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { promote: false } } }
I20240502 14:01:01.700393 8275 consensus_queue.cc:1035] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.701804 404 consensus_queue.cc:237] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:01.703522 3017 raft_consensus.cc:1261] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.704360 673 raft_consensus.cc:1261] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.704684 9385 consensus_queue.cc:1035] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.704947 9385 consensus_queue.cc:1035] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.706758 3017 raft_consensus.cc:2932] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.711509 9160 raft_consensus.cc:2932] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.712441 9385 raft_consensus.cc:2932] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.713419 672 raft_consensus.cc:2932] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.715106 32720 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 604579a2517d4c2eaf855db299d15d5e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.716603 32731 catalog_manager.cc:5484] T 604579a2517d4c2eaf855db299d15d5e P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:01.719693 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet 604579a2517d4c2eaf855db299d15d5e with delete_type TABLET_DATA_TOMBSTONED (TS 2b6796cf0e2e496e9dd641beb432f485 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60258
I20240502 14:01:01.720104 9194 tablet_replica.cc:330] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:01:01.720288 9194 raft_consensus.cc:2229] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.720455 9194 raft_consensus.cc:2258] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.720927 32729 catalog_manager.cc:5484] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.722023 9194 ts_tablet_manager.cc:1875] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.723168 673 raft_consensus.cc:2932] T 604579a2517d4c2eaf855db299d15d5e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.724658 3019 raft_consensus.cc:2932] T 604579a2517d4c2eaf855db299d15d5e P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.726801 2886 raft_consensus.cc:2932] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.729121 803 raft_consensus.cc:2932] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.732368 9194 ts_tablet_manager.cc:1888] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:01.732441 9194 log.cc:1196] T 604579a2517d4c2eaf855db299d15d5e P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/604579a2517d4c2eaf855db299d15d5e
I20240502 14:01:01.732916 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet 604579a2517d4c2eaf855db299d15d5e (table rebalancer_extra_table_46 [id=70b6fe08fece4cc7b481dfd7c0360f3e]) successfully deleted
I20240502 14:01:01.735543 9402 ts_tablet_manager.cc:902] T c80435eecda64ecea7864522f5a1dd4c P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
F20240502 14:01:01.735728 9402 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-2df438b3678241d9a6aa6f0deb2cee2d.queue_time_us, tablet-download-pool-2df438b3678241d9a6aa6f0deb2cee2d.queue_time_us) (tablet-download-pool-2df438b3678241d9a6aa6f0deb2cee2d.run_cpu_time_us, tablet-download-pool-2df438b3678241d9a6aa6f0deb2cee2d.run_cpu_time_us) (tablet-download-pool-2df438b3678241d9a6aa6f0deb2cee2d.run_wall_time_us, tablet-download-pool-2df438b3678241d9a6aa6f0deb2cee2d.run_wall_time_us) (tablet-download-pool-3bcaf05a01194e919efca9cc08d61c8f.queue_time_us, tablet-download-pool-3bcaf05a01194e919efca9cc08d61c8f.queue_time_us) (tablet-download-pool-3bcaf05a01194e919efca9cc08d61c8f.run_cpu_time_us, tablet-download-pool-3bcaf05a01194e919efca9cc08d61c8f.run_cpu_time_us) (tablet-download-pool-3bcaf05a01194e919efca9cc08d61c8f.run_wall_time_us, tablet-download-pool-3bcaf05a01194e919efca9cc08d61c8f.run_wall_time_us) (tablet-download-pool-58a0cf9cd92a43ca8c481964707eda6d.queue_time_us, tablet-download-pool-58a0cf9cd92a43ca8c481964707eda6d.queue_time_us) (tablet-download-pool-58a0cf9cd92a43ca8c481964707eda6d.run_cpu_time_us, tablet-download-pool-58a0cf9cd92a43ca8c481964707eda6d.run_cpu_time_us) (tablet-download-pool-58a0cf9cd92a43ca8c481964707eda6d.run_wall_time_us, tablet-download-pool-58a0cf9cd92a43ca8c481964707eda6d.run_wall_time_us) (tablet-download-pool-5e922f7cee8046e7b4263dd57b8d1185.queue_time_us, tablet-download-pool-5e922f7cee8046e7b4263dd57b8d1185.queue_time_us) (tablet-download-pool-5e922f7cee8046e7b4263dd57b8d1185.run_cpu_time_us, tablet-download-pool-5e922f7cee8046e7b4263dd57b8d1185.run_cpu_time_us) (tablet-download-pool-5e922f7cee8046e7b4263dd57b8d1185.run_wall_time_us, tablet-download-pool-5e922f7cee8046e7b4263dd57b8d1185.run_wall_time_us) (tablet-download-pool-6346d5ea09834bddb483da26f9bce936.queue_time_us, tablet-download-pool-6346d5ea09834bddb483da26f9bce936.queue_time_us) (tablet-download-pool-6346d5ea09834bddb483da26f9bce936.run_cpu_time_us, tablet-download-pool-6346d5ea09834bddb483da26f9bce936.run_cpu_time_us) (tablet-download-pool-6346d5ea09834bddb483da26f9bce936.run_wall_time_us, tablet-download-pool-6346d5ea09834bddb483da26f9bce936.run_wall_time_us) (tablet-download-pool-72590739196d48ac88d69b5bcd1124b0.queue_time_us, tablet-download-pool-72590739196d48ac88d69b5bcd1124b0.queue_time_us) (tablet-download-pool-72590739196d48ac88d69b5bcd1124b0.run_cpu_time_us, tablet-download-pool-72590739196d48ac88d69b5bcd1124b0.run_cpu_time_us) (tablet-download-pool-72590739196d48ac88d69b5bcd1124b0.run_wall_time_us, tablet-download-pool-72590739196d48ac88d69b5bcd1124b0.run_wall_time_us) (tablet-download-pool-81cd2e05295641678fa8d7711c797113.queue_time_us, tablet-download-pool-81cd2e05295641678fa8d7711c797113.queue_time_us) (tablet-download-pool-81cd2e05295641678fa8d7711c797113.run_cpu_time_us, tablet-download-pool-81cd2e05295641678fa8d7711c797113.run_cpu_time_us) (tablet-download-pool-81cd2e05295641678fa8d7711c797113.run_wall_time_us, tablet-download-pool-81cd2e05295641678fa8d7711c797113.run_wall_time_us) (tablet-download-pool-9a9aac11ac0b44339b6d9b0f84702871.queue_time_us, tablet-download-pool-9a9aac11ac0b44339b6d9b0f84702871.queue_time_us) (tablet-download-pool-9a9aac11ac0b44339b6d9b0f84702871.run_cpu_time_us, tablet-download-pool-9a9aac11ac0b44339b6d9b0f84702871.run_cpu_time_us) (tablet-download-pool-9a9aac11ac0b44339b6d9b0f84702871.run_wall_time_us, tablet-download-pool-9a9aac11ac0b44339b6d9b0f84702871.run_wall_time_us) (tablet-download-pool-adaf3fcf84de4223876c7601bdfd45ea.queue_time_us, tablet-download-pool-adaf3fcf84de4223876c7601bdfd45ea.queue_time_us) (tablet-download-pool-adaf3fcf84de4223876c7601bdfd45ea.run_cpu_time_us, tablet-download-pool-adaf3fcf84de4223876c7601bdfd45ea.run_cpu_time_us) (tablet-download-pool-adaf3fcf84de4223876c7601bdfd45ea.run_wall_time_us, tablet-download-pool-adaf3fcf84de4223876c7601bdfd45ea.run_wall_time_us) (tablet-download-pool-b985ba4b9f534827919cbb24daa4de0e.queue_time_us, tablet-download-pool-b985ba4b9f534827919cbb24daa4de0e.queue_time_us) (tablet-download-pool-b985ba4b9f534827919cbb24daa4de0e.run_cpu_time_us, tablet-download-pool-b985ba4b9f534827919cbb24daa4de0e.run_cpu_time_us) (tablet-download-pool-b985ba4b9f534827919cbb24daa4de0e.run_wall_time_us, tablet-download-pool-b985ba4b9f534827919cbb24daa4de0e.run_wall_time_us) (tablet-download-pool-c1dc8816aa514d1abcce63ce0e0bf357.queue_time_us, tablet-download-pool-c1dc8816aa514d1abcce63ce0e0bf357.queue_time_us) (tablet-download-pool-c1dc8816aa514d1abcce63ce0e0bf357.run_cpu_time_us, tablet-download-pool-c1dc8816aa514d1abcce63ce0e0bf357.run_cpu_time_us) (tablet-download-pool-c1dc8816aa514d1abcce63ce0e0bf357.run_wall_time_us, tablet-download-pool-c1dc8816aa514d1abcce63ce0e0bf357.run_wall_time_us) (tablet-download-pool-c80435eecda64ecea7864522f5a1dd4c.queue_time_us, tablet-download-pool-c80435eecda64ecea7864522f5a1dd4c.queue_time_us) (tablet-download-pool-c9d974d536ed41bb9cdc0c3abb64a773.queue_time_us, tablet-download-pool-c9d974d536ed41bb9cdc0c3abb64a773.queue_time_us) (tablet-download-pool-c9d974d536ed41bb9cdc0c3abb64a773.run_cpu_time_us, tablet-download-pool-c9d974d536ed41bb9cdc0c3abb64a773.run_cpu_time_us) (tablet-download-pool-c9d974d536ed41bb9cdc0c3abb64a773.run_wall_time_us, tablet-download-pool-c9d974d536ed41bb9cdc0c3abb64a773.run_wall_time_us) (tablet-download-pool-db0bf9f6821045e1a259703fc3445226.queue_time_us, tablet-download-pool-db0bf9f6821045e1a259703fc3445226.queue_time_us) (tablet-download-pool-db0bf9f6821045e1a259703fc3445226.run_cpu_time_us, tablet-download-pool-db0bf9f6821045e1a259703fc3445226.run_cpu_time_us) (tablet-download-pool-db0bf9f6821045e1a259703fc3445226.run_wall_time_us, tablet-download-pool-db0bf9f6821045e1a259703fc3445226.run_wall_time_us) (tablet-download-pool-f3d9ae2ed9a84c2c928dc1d03f325fa3.queue_time_us, tablet-download-pool-f3d9ae2ed9a84c2c928dc1d03f325fa3.queue_time_us) (tablet-download-pool-f3d9ae2ed9a84c2c928dc1d03f325fa3.run_cpu_time_us, tablet-download-pool-f3d9ae2ed9a84c2c928dc1d03f325fa3.run_cpu_time_us) (tablet-download-pool-f3d9ae2ed9a84c2c928dc1d03f325fa3.run_wall_time_us, tablet-download-pool-f3d9ae2ed9a84c2c928dc1d03f325fa3.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
@ 0x7f531bbcadcd google::LogMessage::Fail() at ??:0
@ 0x7f531bbceb93 google::LogMessage::SendToLog() at ??:0
I20240502 14:01:01.742728 3019 consensus_queue.cc:237] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } }
I20240502 14:01:01.744256 673 raft_consensus.cc:1261] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.744758 803 raft_consensus.cc:1261] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.744742 9160 consensus_queue.cc:1035] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
@ 0x7f531bbca7cc google::LogMessage::Flush() at ??:0
I20240502 14:01:01.745522 9160 consensus_queue.cc:1035] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
@ 0x7f531bbcbf59 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20240502 14:01:01.748282 9160 raft_consensus.cc:2932] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
@ 0x7f531d034044 kudu::TraceMetrics::InternName() at ??:0
I20240502 14:01:01.750792 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 10c70369a07e45afb7bd885b58daec40 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.750795 803 raft_consensus.cc:2932] T 10c70369a07e45afb7bd885b58daec40 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.751716 673 raft_consensus.cc:2932] T 10c70369a07e45afb7bd885b58daec40 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } } }
I20240502 14:01:01.752552 32726 catalog_manager.cc:5484] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index 3 to 4, VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) evicted. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
@ 0x7f531d018f98 kudu::ThreadPool::ThreadPool() at ??:0
I20240502 14:01:01.757067 2866 tablet_service.cc:1514] Processing DeleteTablet for tablet 10c70369a07e45afb7bd885b58daec40 with delete_type TABLET_DATA_TOMBSTONED (TS 4bcd7712aa844d98977a7d7e605c31f5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41400
I20240502 14:01:01.757313 9101 tablet_replica.cc:330] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5: stopping tablet replica
I20240502 14:01:01.757496 9101 raft_consensus.cc:2229] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.757661 9101 raft_consensus.cc:2258] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.759053 9101 ts_tablet_manager.cc:1875] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
@ 0x7f531d016efc kudu::ThreadPoolBuilder::Build() at ??:0
I20240502 14:01:01.761585 9101 ts_tablet_manager.cc:1888] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:01.761648 9101 log.cc:1196] T 10c70369a07e45afb7bd885b58daec40 P 4bcd7712aa844d98977a7d7e605c31f5: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/wals/10c70369a07e45afb7bd885b58daec40
I20240502 14:01:01.762617 32721 catalog_manager.cc:4830] TS 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): tablet 10c70369a07e45afb7bd885b58daec40 (table rebalancer_extra_table_2 [id=15d9dbc663c942979a768e467c809991]) successfully deleted
I20240502 14:01:01.764037 8525 raft_consensus.cc:1050] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01: attempting to promote NON_VOTER da3f8160557e454f8547f45e80c2f013 to VOTER
I20240502 14:01:01.764511 8525 consensus_queue.cc:237] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
@ 0x7f531fbc977a kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
I20240502 14:01:01.765975 3283 raft_consensus.cc:1261] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.766422 3152 raft_consensus.cc:1261] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.766583 9193 consensus_queue.cc:1035] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.766810 9014 consensus_queue.cc:1035] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.768530 540 raft_consensus.cc:1261] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.769213 9014 consensus_queue.cc:1035] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
@ 0x7f531fbd2a45 kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
I20240502 14:01:01.772420 9014 raft_consensus.cc:2932] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:01:01.775164 540 raft_consensus.cc:2932] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
@ 0x7f531fc5376f kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
I20240502 14:01:01.777264 3283 raft_consensus.cc:2932] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:01:01.777235 3152 raft_consensus.cc:2932] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:01:01.780658 32726 catalog_manager.cc:5484] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 reported cstate change: config changed from index 2 to 3, da3f8160557e454f8547f45e80c2f013 (127.26.25.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
@ 0x7f531fc674f6 kudu::tserver::TabletCopyRunnable::Run() at ??:0
@ 0x7f531fc52226 _ZZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS_6StatusENS0_24TabletServerErrorPB_CodeEEEENKUlvE_clEv at ??:0
@ 0x7f531fc61c0a _ZNSt17_Function_handlerIFvvEZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS1_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS1_6StatusENS2_24TabletServerErrorPB_CodeEEEEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x5583eee0775c std::function<>::operator()() at ??:0
I20240502 14:01:01.791004 9422 ts_tablet_manager.cc:902] T 20937116074f456ea732396f80bd7d6a P 478dfa53d7944f38b4e6cabee21b2d01: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:01:01.791401 9421 ts_tablet_manager.cc:902] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:01:01.792094 9421 tablet_copy_client.cc:308] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
@ 0x7f531d01be06 kudu::ThreadPool::DispatchThread() at ??:0
I20240502 14:01:01.792578 2905 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet d493cec70a39401b98c21446f866789a from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:33043)
I20240502 14:01:01.792678 2905 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet d493cec70a39401b98c21446f866789a from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:33043: session id = 2b6796cf0e2e496e9dd641beb432f485-d493cec70a39401b98c21446f866789a
I20240502 14:01:01.793752 2905 tablet_copy_source_session.cc:205] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:01.794894 9421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d493cec70a39401b98c21446f866789a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:01.797264 803 consensus_queue.cc:237] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
@ 0x7f531d01c6ef _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
I20240502 14:01:01.798476 3283 raft_consensus.cc:1261] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.798956 6174 consensus_queue.cc:1035] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.799861 3152 raft_consensus.cc:1261] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 478dfa53d7944f38b4e6cabee21b2d01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:01.800141 9421 tablet_copy_client.cc:783] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:01.800315 9421 tablet_copy_client.cc:647] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
@ 0x7f531d01e03b _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
I20240502 14:01:01.801417 9193 raft_consensus.cc:2932] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
@ 0x5583eee0775c std::function<>::operator()() at ??:0
I20240502 14:01:01.801751 3283 raft_consensus.cc:2932] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:01:01.803092 9193 consensus_queue.cc:1035] T 167e1eb3a4034f3ca75e721b8140d757 P 478dfa53d7944f38b4e6cabee21b2d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:01.804051 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 167e1eb3a4034f3ca75e721b8140d757 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:01.804355 9421 tablet_copy_client.cc:523] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
@ 0x7f531d00b807 kudu::Thread::SuperviseThread() at ??:0
I20240502 14:01:01.806337 9421 tablet_bootstrap.cc:492] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:01:01.807471 32732 catalog_manager.cc:5484] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New cstate: current_term: 1 leader_uuid: "478dfa53d7944f38b4e6cabee21b2d01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
@ 0x7f531d8496db start_thread at ??:0
@ 0x7f531ad8171f clone at ??:0
I20240502 14:01:01.811106 9421 tablet_bootstrap.cc:492] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:01.811352 9421 tablet_bootstrap.cc:492] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:01:01.811753 9421 ts_tablet_manager.cc:1367] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20240502 14:01:01.812206 9421 raft_consensus.cc:348] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:01:01.812321 9421 raft_consensus.cc:729] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:01:01.812425 9421 consensus_queue.cc:260] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:01:01.813318 3150 raft_consensus.cc:2932] T 167e1eb3a4034f3ca75e721b8140d757 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } } }
I20240502 14:01:01.814239 9421 ts_tablet_manager.cc:1398] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:01.815038 2905 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-d493cec70a39401b98c21446f866789a received from {username='slave'} at 127.26.25.135:33043
I20240502 14:01:01.815115 2905 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-d493cec70a39401b98c21446f866789a on tablet d493cec70a39401b98c21446f866789a with peer 2b6796cf0e2e496e9dd641beb432f485
I20240502 14:01:01.815296 520 tablet_service.cc:1514] Processing DeleteTablet for tablet 167e1eb3a4034f3ca75e721b8140d757 with delete_type TABLET_DATA_TOMBSTONED (TS c351a89a5fe34e788b9d8064fbffa751 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60890
I20240502 14:01:01.816073 9434 tablet_replica.cc:330] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751: stopping tablet replica
I20240502 14:01:01.816217 9434 raft_consensus.cc:2229] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:01.816363 9434 raft_consensus.cc:2258] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:01.817127 9434 ts_tablet_manager.cc:1875] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:01.819584 9434 ts_tablet_manager.cc:1888] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:01.819649 9434 log.cc:1196] T 167e1eb3a4034f3ca75e721b8140d757 P c351a89a5fe34e788b9d8064fbffa751: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/wals/167e1eb3a4034f3ca75e721b8140d757
I20240502 14:01:01.821264 32718 catalog_manager.cc:4830] TS c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): tablet 167e1eb3a4034f3ca75e721b8140d757 (table rebalancer_extra_table_66 [id=cb41deb677b547cd82087726509f0f21]) successfully deleted
I20240502 14:01:01.869616 8966 raft_consensus.cc:1050] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485: attempting to promote NON_VOTER f003b732dd0f46958262b42f8d20305e to VOTER
I20240502 14:01:01.870061 8966 consensus_queue.cc:237] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:01:01.871585 404 raft_consensus.cc:1261] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.871994 7246 consensus_queue.cc:1035] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.872594 540 raft_consensus.cc:1261] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:01.872985 7246 consensus_queue.cc:1035] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:01.874804 7246 raft_consensus.cc:2932] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:01.877205 404 raft_consensus.cc:2932] T 7450ad74adcc49c3820f93b568ee7c21 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:01.877316 540 raft_consensus.cc:2932] T 7450ad74adcc49c3820f93b568ee7c21 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:01.880105 32732 catalog_manager.cc:5484] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 2 to 3, f003b732dd0f46958262b42f8d20305e (127.26.25.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:01.897639 9464 ts_tablet_manager.cc:902] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:01:01.898241 9464 tablet_copy_client.cc:308] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:01:01.898684 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet 527784cdca2f4840aa2f2a373a2c6db6 from peer 452ecbc085e54db49c3b01c20a96f860 ({username='slave'} at 127.26.25.134:46455)
I20240502 14:01:01.898808 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet 527784cdca2f4840aa2f2a373a2c6db6 from peer 452ecbc085e54db49c3b01c20a96f860 at {username='slave'} at 127.26.25.134:46455: session id = 452ecbc085e54db49c3b01c20a96f860-527784cdca2f4840aa2f2a373a2c6db6
I20240502 14:01:01.900022 3172 tablet_copy_source_session.cc:205] T 527784cdca2f4840aa2f2a373a2c6db6 P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:01.901279 9464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 527784cdca2f4840aa2f2a373a2c6db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:01.903970 9464 tablet_copy_client.cc:783] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:01.904134 9464 tablet_copy_client.cc:647] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:01.905161 9464 tablet_copy_client.cc:523] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:01.906584 9464 tablet_bootstrap.cc:492] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:01:01.910677 9464 tablet_bootstrap.cc:492] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:01.910946 9464 tablet_bootstrap.cc:492] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860: Bootstrap complete.
I20240502 14:01:01.911118 9464 ts_tablet_manager.cc:1367] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 14:01:01.911617 9464 raft_consensus.cc:348] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:01:01.911751 9464 raft_consensus.cc:729] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: LEARNER
I20240502 14:01:01.911911 9464 consensus_queue.cc:260] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:01:01.912211 9464 ts_tablet_manager.cc:1398] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:01.912724 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session 452ecbc085e54db49c3b01c20a96f860-527784cdca2f4840aa2f2a373a2c6db6 received from {username='slave'} at 127.26.25.134:46455
I20240502 14:01:01.912813 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session 452ecbc085e54db49c3b01c20a96f860-527784cdca2f4840aa2f2a373a2c6db6 on tablet 527784cdca2f4840aa2f2a373a2c6db6 with peer 452ecbc085e54db49c3b01c20a96f860
I20240502 14:01:01.939993 9358 ts_tablet_manager.cc:902] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:01:01.940521 9358 tablet_copy_client.cc:308] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:01:01.941025 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet 8689975c1d9d41f4ae7152bf6e4a58a8 from peer 4bcd7712aa844d98977a7d7e605c31f5 ({username='slave'} at 127.26.25.133:46031)
I20240502 14:01:01.941149 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet 8689975c1d9d41f4ae7152bf6e4a58a8 from peer 4bcd7712aa844d98977a7d7e605c31f5 at {username='slave'} at 127.26.25.133:46031: session id = 4bcd7712aa844d98977a7d7e605c31f5-8689975c1d9d41f4ae7152bf6e4a58a8
I20240502 14:01:01.942339 693 tablet_copy_source_session.cc:205] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:01.943586 9358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8689975c1d9d41f4ae7152bf6e4a58a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:01.946133 9358 tablet_copy_client.cc:783] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:01.946316 9358 tablet_copy_client.cc:647] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:01.947491 9358 tablet_copy_client.cc:523] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:01.948808 9358 tablet_bootstrap.cc:492] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:01:01.971910 9358 tablet_bootstrap.cc:492] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:01.972301 9358 tablet_bootstrap.cc:492] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap complete.
I20240502 14:01:01.972496 9358 ts_tablet_manager.cc:1367] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.024s user 0.023s sys 0.000s
I20240502 14:01:01.973029 9358 raft_consensus.cc:348] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:01:01.973155 9358 raft_consensus.cc:729] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: LEARNER
I20240502 14:01:01.973279 9358 consensus_queue.cc:260] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:01:01.974681 9358 ts_tablet_manager.cc:1398] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:01.975157 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-8689975c1d9d41f4ae7152bf6e4a58a8 received from {username='slave'} at 127.26.25.133:46031
I20240502 14:01:01.975250 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-8689975c1d9d41f4ae7152bf6e4a58a8 on tablet 8689975c1d9d41f4ae7152bf6e4a58a8 with peer 4bcd7712aa844d98977a7d7e605c31f5
I20240502 14:01:02.001658 9475 ts_tablet_manager.cc:902] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751: Initiating tablet copy from peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:01:02.002264 9475 tablet_copy_client.cc:308] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.133:34751
I20240502 14:01:02.002784 2905 tablet_copy_service.cc:140] P 4bcd7712aa844d98977a7d7e605c31f5: Received BeginTabletCopySession request for tablet 0dc1880244a64b9f96ad5b8122a49fce from peer c351a89a5fe34e788b9d8064fbffa751 ({username='slave'} at 127.26.25.130:46819)
I20240502 14:01:02.002895 2905 tablet_copy_service.cc:161] P 4bcd7712aa844d98977a7d7e605c31f5: Beginning new tablet copy session on tablet 0dc1880244a64b9f96ad5b8122a49fce from peer c351a89a5fe34e788b9d8064fbffa751 at {username='slave'} at 127.26.25.130:46819: session id = c351a89a5fe34e788b9d8064fbffa751-0dc1880244a64b9f96ad5b8122a49fce
I20240502 14:01:02.003928 2905 tablet_copy_source_session.cc:205] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:02.007067 9475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc1880244a64b9f96ad5b8122a49fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:02.010181 9475 tablet_copy_client.cc:783] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:02.010402 9475 tablet_copy_client.cc:647] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:02.011653 9475 tablet_copy_client.cc:523] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:02.012869 9475 tablet_bootstrap.cc:492] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751: Bootstrap starting.
I20240502 14:01:02.016746 2886 raft_consensus.cc:1203] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:02.017050 9475 tablet_bootstrap.cc:492] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:02.017319 9475 tablet_bootstrap.cc:492] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751: Bootstrap complete.
I20240502 14:01:02.017517 9475 ts_tablet_manager.cc:1367] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20240502 14:01:02.017966 9475 raft_consensus.cc:348] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:01:02.018142 9475 raft_consensus.cc:729] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Initialized, Role: LEARNER
I20240502 14:01:02.018246 9475 consensus_queue.cc:260] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: NON_VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: true } }
I20240502 14:01:02.018512 9475 ts_tablet_manager.cc:1398] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:02.020817 2905 tablet_copy_service.cc:342] P 4bcd7712aa844d98977a7d7e605c31f5: Request end of tablet copy session c351a89a5fe34e788b9d8064fbffa751-0dc1880244a64b9f96ad5b8122a49fce received from {username='slave'} at 127.26.25.130:46819
I20240502 14:01:02.020906 2905 tablet_copy_service.cc:433] P 4bcd7712aa844d98977a7d7e605c31f5: ending tablet copy session c351a89a5fe34e788b9d8064fbffa751-0dc1880244a64b9f96ad5b8122a49fce on tablet 0dc1880244a64b9f96ad5b8122a49fce with peer c351a89a5fe34e788b9d8064fbffa751
I20240502 14:01:02.025173 9464 ts_tablet_manager.cc:902] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:01:02.025628 9464 tablet_copy_client.cc:308] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:01:02.026204 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet 9bf2e05fba1943edb5cd00989f93df7c from peer 452ecbc085e54db49c3b01c20a96f860 ({username='slave'} at 127.26.25.134:46455)
I20240502 14:01:02.026310 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet 9bf2e05fba1943edb5cd00989f93df7c from peer 452ecbc085e54db49c3b01c20a96f860 at {username='slave'} at 127.26.25.134:46455: session id = 452ecbc085e54db49c3b01c20a96f860-9bf2e05fba1943edb5cd00989f93df7c
I20240502 14:01:02.027083 3172 tablet_copy_source_session.cc:205] T 9bf2e05fba1943edb5cd00989f93df7c P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:02.027940 9464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bf2e05fba1943edb5cd00989f93df7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:02.030140 9464 tablet_copy_client.cc:783] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:02.030290 9464 tablet_copy_client.cc:647] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:02.033574 540 consensus_queue.cc:237] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:01:02.034869 405 raft_consensus.cc:1261] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:02.035326 9320 consensus_queue.cc:1035] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:02.035393 3017 raft_consensus.cc:1261] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:02.035779 9130 consensus_queue.cc:1035] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:02.036883 9464 tablet_copy_client.cc:523] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:02.037830 9130 raft_consensus.cc:2932] T c2ce30696a474d23818c3f70cc86e9dd P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:02.038231 9464 tablet_bootstrap.cc:492] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860: Bootstrap starting.
I20240502 14:01:02.038877 3017 raft_consensus.cc:2932] T c2ce30696a474d23818c3f70cc86e9dd P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:02.040773 405 raft_consensus.cc:2932] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:02.044207 32731 catalog_manager.cc:5484] T c2ce30696a474d23818c3f70cc86e9dd P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index 3 to 4, VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) evicted. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } } }
I20240502 14:01:02.044852 9464 tablet_bootstrap.cc:492] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:02.045228 9464 tablet_bootstrap.cc:492] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860: Bootstrap complete.
I20240502 14:01:02.045390 9464 ts_tablet_manager.cc:1367] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20240502 14:01:02.046563 9464 raft_consensus.cc:348] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:01:02.046684 9464 raft_consensus.cc:729] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Initialized, Role: LEARNER
I20240502 14:01:02.046780 9464 consensus_queue.cc:260] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: NON_VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: true } }
I20240502 14:01:02.047119 9464 ts_tablet_manager.cc:1398] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 14:01:02.051030 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session 452ecbc085e54db49c3b01c20a96f860-9bf2e05fba1943edb5cd00989f93df7c received from {username='slave'} at 127.26.25.134:46455
I20240502 14:01:02.051119 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session 452ecbc085e54db49c3b01c20a96f860-9bf2e05fba1943edb5cd00989f93df7c on tablet 9bf2e05fba1943edb5cd00989f93df7c with peer 452ecbc085e54db49c3b01c20a96f860
I20240502 14:01:02.051123 3132 tablet_service.cc:1514] Processing DeleteTablet for tablet c2ce30696a474d23818c3f70cc86e9dd with delete_type TABLET_DATA_TOMBSTONED (TS 2b6796cf0e2e496e9dd641beb432f485 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60258
I20240502 14:01:02.051518 9194 tablet_replica.cc:330] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485: stopping tablet replica
I20240502 14:01:02.051694 9194 raft_consensus.cc:2229] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:02.051851 9194 raft_consensus.cc:2258] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:02.052723 9194 ts_tablet_manager.cc:1875] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:02.053045 32718 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c2ce30696a474d23818c3f70cc86e9dd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:02.054987 9194 ts_tablet_manager.cc:1888] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:02.055068 9194 log.cc:1196] T c2ce30696a474d23818c3f70cc86e9dd P 2b6796cf0e2e496e9dd641beb432f485: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/wals/c2ce30696a474d23818c3f70cc86e9dd
I20240502 14:01:02.062238 32721 catalog_manager.cc:4830] TS 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): tablet c2ce30696a474d23818c3f70cc86e9dd (table rebalancer_extra_table_4 [id=13b33238c8214cd6b88686b2b94abdb2]) successfully deleted
I20240502 14:01:02.070782 9421 ts_tablet_manager.cc:902] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:01:02.071295 9421 tablet_copy_client.cc:308] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:01:02.071766 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet c8e843cd7c984ebfb532fec67f97e249 from peer 2b6796cf0e2e496e9dd641beb432f485 ({username='slave'} at 127.26.25.135:37651)
I20240502 14:01:02.071923 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet c8e843cd7c984ebfb532fec67f97e249 from peer 2b6796cf0e2e496e9dd641beb432f485 at {username='slave'} at 127.26.25.135:37651: session id = 2b6796cf0e2e496e9dd641beb432f485-c8e843cd7c984ebfb532fec67f97e249
I20240502 14:01:02.072935 693 tablet_copy_source_session.cc:205] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:02.073807 9421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8e843cd7c984ebfb532fec67f97e249. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:02.076334 9421 tablet_copy_client.cc:783] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:02.076483 9421 tablet_copy_client.cc:647] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:02.083634 9421 tablet_copy_client.cc:523] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:02.084900 9421 tablet_bootstrap.cc:492] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap starting.
I20240502 14:01:02.090350 9421 tablet_bootstrap.cc:492] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:02.090586 9421 tablet_bootstrap.cc:492] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485: Bootstrap complete.
I20240502 14:01:02.090803 9421 ts_tablet_manager.cc:1367] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20240502 14:01:02.091246 9421 raft_consensus.cc:348] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:01:02.091357 9421 raft_consensus.cc:729] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b6796cf0e2e496e9dd641beb432f485, State: Initialized, Role: LEARNER
I20240502 14:01:02.091447 9421 consensus_queue.cc:260] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:01:02.091727 9421 ts_tablet_manager.cc:1398] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20240502 14:01:02.091792 2954 connection.cc:535] client connection to 127.26.25.132:33443 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.091941 3084 connection.cc:535] server connection from 127.26.25.132:57613 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.091998 2954 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.092614 3087 connection.cc:535] client connection to 127.26.25.132:33443 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.092660 3219 connection.cc:535] client connection to 127.26.25.132:33443 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.092726 3087 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.092753 3219 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.092883 609 connection.cc:535] client connection to 127.26.25.132:33443 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.093004 609 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.093046 2822 connection.cc:535] client connection to 127.26.25.132:33443 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.093161 2822 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.093362 477 connection.cc:535] client connection to 127.26.25.132:33443 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:02.093479 477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 14:01:02.093938 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session 2b6796cf0e2e496e9dd641beb432f485-c8e843cd7c984ebfb532fec67f97e249 received from {username='slave'} at 127.26.25.135:37651
I20240502 14:01:02.094017 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session 2b6796cf0e2e496e9dd641beb432f485-c8e843cd7c984ebfb532fec67f97e249 on tablet c8e843cd7c984ebfb532fec67f97e249 with peer 2b6796cf0e2e496e9dd641beb432f485
W20240502 14:01:02.095961 477 consensus_peers.cc:485] T 2df438b3678241d9a6aa6f0deb2cee2d P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.096033 477 consensus_peers.cc:485] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.096067 477 consensus_peers.cc:485] T 76f41f7f19c94930aaba452854be2aaa P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.096694 3219 consensus_peers.cc:485] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.096760 3219 consensus_peers.cc:485] T b985ba4b9f534827919cbb24daa4de0e P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.097676 2954 consensus_peers.cc:485] T 638cb0f08414491ea80139eb51406dfe P 452ecbc085e54db49c3b01c20a96f860 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.097748 2954 consensus_peers.cc:485] T 909bc2eb63544db0b4b3a0764be7d4a1 P 452ecbc085e54db49c3b01c20a96f860 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.097805 2954 consensus_peers.cc:485] T 6ab1a0788bee474e8508bf71bb9a6373 P 452ecbc085e54db49c3b01c20a96f860 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.098500 2822 consensus_peers.cc:485] T 9e642731d7684e33b82eabf97c380123 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.098574 2822 consensus_peers.cc:485] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.098641 2822 consensus_peers.cc:485] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.099061 3219 consensus_peers.cc:485] T 0199aa852916433dafaf8a08720ca6d3 P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.099131 3219 consensus_peers.cc:485] T 6346d5ea09834bddb483da26f9bce936 P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.099169 3219 consensus_peers.cc:462] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
W20240502 14:01:02.100337 609 consensus_peers.cc:485] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.100417 609 consensus_peers.cc:485] T 567df9d664b7423785175fa385f83e53 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.100468 609 consensus_peers.cc:485] T d60b2e45a34e4d8f95b33e219d75e78f P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.100513 609 consensus_peers.cc:485] T d37a7cd2db2a431caaba0257f9534e3b P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.100559 609 consensus_peers.cc:485] T ce8c8f96e8944bba88ad9b818e27c916 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.100620 609 consensus_peers.cc:485] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.100666 609 consensus_peers.cc:485] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.101090 3087 consensus_peers.cc:485] T c9d974d536ed41bb9cdc0c3abb64a773 P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.101150 3087 consensus_peers.cc:485] T 7450ad74adcc49c3820f93b568ee7c21 P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.101174 3087 consensus_peers.cc:462] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
W20240502 14:01:02.106089 609 consensus_peers.cc:485] T 7e9f86a6149d43f3a217769865d9a3d9 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.113835 477 consensus_peers.cc:485] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.115728 2886 raft_consensus.cc:1203] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
W20240502 14:01:02.129305 609 consensus_peers.cc:485] T 72590739196d48ac88d69b5bcd1124b0 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.133478 609 consensus_peers.cc:485] T 9a9aac11ac0b44339b6d9b0f84702871 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.145977 2954 consensus_peers.cc:485] T 10c70369a07e45afb7bd885b58daec40 P 452ecbc085e54db49c3b01c20a96f860 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.156479 3219 consensus_peers.cc:485] T be10cc2fc95c475f8bb0868229f748f4 P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.163296 344 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
W20240502 14:01:02.164662 344 consensus_peers.cc:485] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.168812 541 consensus_queue.cc:237] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } }
I20240502 14:01:02.170251 673 raft_consensus.cc:1261] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:01:02.170503 477 consensus_peers.cc:485] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1415dfd1daed4e1a8a00439c77185a55. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.170866 9320 consensus_queue.cc:1035] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:01:02.171030 477 consensus_peers.cc:485] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.171754 609 consensus_peers.cc:485] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.172631 9487 raft_consensus.cc:2932] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:01:02.174338 673 raft_consensus.cc:2932] T 1415dfd1daed4e1a8a00439c77185a55 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } }
I20240502 14:01:02.176759 32732 catalog_manager.cc:5484] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index -1 to 4, NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) added. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:02.188882 3087 consensus_peers.cc:462] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.198140 3150 consensus_queue.cc:237] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:02.199779 409 raft_consensus.cc:1261] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:02.199846 3284 raft_consensus.cc:1261] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 2b6796cf0e2e496e9dd641beb432f485: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:01:02.199885 3084 consensus_peers.cc:485] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 544ed46acc5345f58031444b9a2ef776. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.200290 8159 consensus_queue.cc:1035] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:02.200506 9457 consensus_queue.cc:1035] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:02.202297 8159 raft_consensus.cc:2932] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
W20240502 14:01:02.203567 3219 consensus_peers.cc:462] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.205590 3284 raft_consensus.cc:2932] T 544ed46acc5345f58031444b9a2ef776 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:02.206043 409 raft_consensus.cc:2932] T 544ed46acc5345f58031444b9a2ef776 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:02.206544 3152 raft_consensus.cc:1203] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20240502 14:01:02.209271 3219 consensus_peers.cc:485] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.213915 477 consensus_peers.cc:485] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.207731 32731 catalog_manager.cc:5484] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "2b6796cf0e2e496e9dd641beb432f485" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:02.220342 2822 consensus_peers.cc:485] T 58a0cf9cd92a43ca8c481964707eda6d P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.224820 673 consensus_queue.cc:237] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:01:02.228404 541 raft_consensus.cc:1261] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:01:02.228626 609 consensus_peers.cc:485] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.228874 6065 consensus_queue.cc:1035] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:01:02.229319 344 consensus_peers.cc:485] T b8afd1dff8f040eb802bc7b1a4b4a26a P f003b732dd0f46958262b42f8d20305e -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.230193 607 consensus_peers.cc:485] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131): Couldn't send request to peer da3f8160557e454f8547f45e80c2f013. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a35103d97b874010b36c1863b2258c33. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.231346 6065 raft_consensus.cc:2932] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
I20240502 14:01:02.232801 540 raft_consensus.cc:2932] T a35103d97b874010b36c1863b2258c33 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } } }
W20240502 14:01:02.238711 3087 consensus_peers.cc:485] T 3bcaf05a01194e919efca9cc08d61c8f P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.238631 32729 catalog_manager.cc:5484] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index -1 to 4, NON_VOTER da3f8160557e454f8547f45e80c2f013 (127.26.25.136) added. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:02.245515 3087 consensus_peers.cc:485] T adaf3fcf84de4223876c7601bdfd45ea P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.249487 2822 consensus_peers.cc:485] T 2b4469ea792f4c1f9dce98e47b51f9f4 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:02.259606 609 consensus_peers.cc:485] T 49fa0956a8a74ee087bbeb605942e219 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.259585 409 consensus_queue.cc:237] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:01:02.261494 540 raft_consensus.cc:1261] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:02.261893 9590 consensus_queue.cc:1035] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:02.262748 673 raft_consensus.cc:1261] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer f003b732dd0f46958262b42f8d20305e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:02.263890 9391 raft_consensus.cc:2932] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
W20240502 14:01:02.265259 344 consensus_peers.cc:485] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e -> Peer 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751): Couldn't send request to peer 4bcd7712aa844d98977a7d7e605c31f5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 02a36ffbdd0f4d68953aa441c04b42eb. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.266268 9390 consensus_queue.cc:1035] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:01:02.268255 477 consensus_peers.cc:485] T ad4e1b6dea1e4beeb4b82852ded83c5f P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.268281 32731 catalog_manager.cc:5484] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e reported cstate change: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New cstate: current_term: 1 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:02.280144 672 raft_consensus.cc:2932] T 02a36ffbdd0f4d68953aa441c04b42eb P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
W20240502 14:01:02.283689 3219 consensus_peers.cc:485] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.285790 540 raft_consensus.cc:2932] T 02a36ffbdd0f4d68953aa441c04b42eb P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } } }
I20240502 14:01:02.292747 3019 consensus_queue.cc:237] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:01:02.294879 540 raft_consensus.cc:1261] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:01:02.295015 2953 consensus_peers.cc:485] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1eab750f98a6432494fa5351269e3052. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.295522 5282 consensus_queue.cc:1035] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:02.296298 3284 raft_consensus.cc:1261] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:02.296847 5282 consensus_queue.cc:1035] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:02.300577 9160 raft_consensus.cc:2932] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:01:02.301092 540 raft_consensus.cc:2932] T 1eab750f98a6432494fa5351269e3052 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
W20240502 14:01:02.305083 477 consensus_peers.cc:485] T 4701a1452cc3428894e0c3a672768b74 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.305847 32732 catalog_manager.cc:5484] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:02.306777 3284 raft_consensus.cc:2932] T 1eab750f98a6432494fa5351269e3052 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER f003b732dd0f46958262b42f8d20305e (127.26.25.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } } }
I20240502 14:01:02.318509 3019 consensus_queue.cc:237] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:02.320032 409 raft_consensus.cc:1261] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 14:01:02.320106 2951 consensus_peers.cc:485] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 844d8895be4b4635bc21650b60049409. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.320510 3150 raft_consensus.cc:1261] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 452ecbc085e54db49c3b01c20a96f860: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:02.320945 5282 consensus_queue.cc:1035] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:02.321133 9393 consensus_queue.cc:1035] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:02.324277 5282 raft_consensus.cc:2932] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:02.325250 409 raft_consensus.cc:2932] T 844d8895be4b4635bc21650b60049409 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:02.327657 3150 raft_consensus.cc:2932] T 844d8895be4b4635bc21650b60049409 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } } }
I20240502 14:01:02.331781 32731 catalog_manager.cc:5484] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: config changed from index -1 to 2, NON_VOTER 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131) added. New cstate: current_term: 1 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:02.351733 3019 raft_consensus.cc:1203] T 527784cdca2f4840aa2f2a373a2c6db6 P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20240502 14:01:02.394913 2822 consensus_peers.cc:485] T 5e922f7cee8046e7b4263dd57b8d1185 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.478828 2885 raft_consensus.cc:1203] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:02.489872 541 raft_consensus.cc:1203] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:02.525624 3019 raft_consensus.cc:1203] T 9bf2e05fba1943edb5cd00989f93df7c P 452ecbc085e54db49c3b01c20a96f860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:02.545197 9081 raft_consensus.cc:1050] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 to VOTER
I20240502 14:01:02.545635 9081 consensus_queue.cc:237] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:01:02.547127 2886 raft_consensus.cc:1261] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:02.547520 6065 consensus_queue.cc:1035] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:02.547662 541 raft_consensus.cc:1261] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 14:01:02.547940 609 consensus_peers.cc:485] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.548130 6065 consensus_queue.cc:1035] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:02.549993 9583 raft_consensus.cc:2932] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:02.550364 541 raft_consensus.cc:2932] T f47f72fcaaae46d68f4cdab7595095ca P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:02.554018 32729 catalog_manager.cc:5484] T f47f72fcaaae46d68f4cdab7595095ca P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:02.554821 2886 raft_consensus.cc:2932] T f47f72fcaaae46d68f4cdab7595095ca P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:02.583254 3150 raft_consensus.cc:1203] T c8e843cd7c984ebfb532fec67f97e249 P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20240502 14:01:02.596081 3087 consensus_peers.cc:462] T 20937116074f456ea732396f80bd7d6a P 2b6796cf0e2e496e9dd641beb432f485 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.610672 9707 ts_tablet_manager.cc:902] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9: Initiating tablet copy from peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:01:02.611174 9707 tablet_copy_client.cc:308] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.135:45509
I20240502 14:01:02.611610 3172 tablet_copy_service.cc:140] P 2b6796cf0e2e496e9dd641beb432f485: Received BeginTabletCopySession request for tablet 544ed46acc5345f58031444b9a2ef776 from peer 58107b611f8f475a910a6fd54dfd1fb9 ({username='slave'} at 127.26.25.131:33211)
I20240502 14:01:02.611728 3172 tablet_copy_service.cc:161] P 2b6796cf0e2e496e9dd641beb432f485: Beginning new tablet copy session on tablet 544ed46acc5345f58031444b9a2ef776 from peer 58107b611f8f475a910a6fd54dfd1fb9 at {username='slave'} at 127.26.25.131:33211: session id = 58107b611f8f475a910a6fd54dfd1fb9-544ed46acc5345f58031444b9a2ef776
I20240502 14:01:02.612813 9320 raft_consensus.cc:1050] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751: attempting to promote NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 to VOTER
I20240502 14:01:02.612843 3172 tablet_copy_source_session.cc:205] T 544ed46acc5345f58031444b9a2ef776 P 2b6796cf0e2e496e9dd641beb432f485: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:02.613232 9320 consensus_queue.cc:237] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:01:02.615024 2886 raft_consensus.cc:1261] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 14:01:02.615370 477 consensus_peers.cc:485] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:02.615489 9485 consensus_queue.cc:1035] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:02.617000 9707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 544ed46acc5345f58031444b9a2ef776. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:02.619338 9707 tablet_copy_client.cc:783] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:02.619488 672 raft_consensus.cc:1261] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:02.619550 9707 tablet_copy_client.cc:647] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:02.620069 9485 consensus_queue.cc:1035] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:02.621799 9485 raft_consensus.cc:2932] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:02.625487 32729 catalog_manager.cc:5484] T 556b3a4cab9245d4939ba4c10bf9d4b4 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:02.625855 672 raft_consensus.cc:2932] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:02.628515 2886 raft_consensus.cc:2932] T 556b3a4cab9245d4939ba4c10bf9d4b4 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:02.631229 9707 tablet_copy_client.cc:523] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:02.632797 9707 tablet_bootstrap.cc:492] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
W20240502 14:01:02.633297 3219 consensus_peers.cc:462] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.638604 9707 tablet_bootstrap.cc:492] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:02.638906 9707 tablet_bootstrap.cc:492] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap complete.
I20240502 14:01:02.639058 9707 ts_tablet_manager.cc:1367] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 14:01:02.639496 9707 raft_consensus.cc:348] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:02.639597 9707 raft_consensus.cc:729] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: LEARNER
I20240502 14:01:02.639717 9707 consensus_queue.cc:260] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { replace: true } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:02.640022 9707 ts_tablet_manager.cc:1398] T 544ed46acc5345f58031444b9a2ef776 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:02.640496 3172 tablet_copy_service.cc:342] P 2b6796cf0e2e496e9dd641beb432f485: Request end of tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-544ed46acc5345f58031444b9a2ef776 received from {username='slave'} at 127.26.25.131:33211
I20240502 14:01:02.640570 3172 tablet_copy_service.cc:433] P 2b6796cf0e2e496e9dd641beb432f485: ending tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-544ed46acc5345f58031444b9a2ef776 on tablet 544ed46acc5345f58031444b9a2ef776 with peer 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:01:02.668089 9717 ts_tablet_manager.cc:902] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5: Initiating tablet copy from peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:01:02.668649 9717 tablet_copy_client.cc:308] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.129:44715
I20240502 14:01:02.669134 429 tablet_copy_service.cc:140] P f003b732dd0f46958262b42f8d20305e: Received BeginTabletCopySession request for tablet 02a36ffbdd0f4d68953aa441c04b42eb from peer 4bcd7712aa844d98977a7d7e605c31f5 ({username='slave'} at 127.26.25.133:44833)
I20240502 14:01:02.669253 429 tablet_copy_service.cc:161] P f003b732dd0f46958262b42f8d20305e: Beginning new tablet copy session on tablet 02a36ffbdd0f4d68953aa441c04b42eb from peer 4bcd7712aa844d98977a7d7e605c31f5 at {username='slave'} at 127.26.25.133:44833: session id = 4bcd7712aa844d98977a7d7e605c31f5-02a36ffbdd0f4d68953aa441c04b42eb
I20240502 14:01:02.670089 429 tablet_copy_source_session.cc:205] T 02a36ffbdd0f4d68953aa441c04b42eb P f003b732dd0f46958262b42f8d20305e: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:02.670989 9717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a36ffbdd0f4d68953aa441c04b42eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:02.673426 9717 tablet_copy_client.cc:783] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:02.673578 9717 tablet_copy_client.cc:647] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:02.679522 9717 tablet_copy_client.cc:523] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:02.680707 9717 tablet_bootstrap.cc:492] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap starting.
I20240502 14:01:02.683995 9722 ts_tablet_manager.cc:902] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013: Initiating tablet copy from peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887)
I20240502 14:01:02.684511 9722 tablet_copy_client.cc:308] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.131:38887
I20240502 14:01:02.684967 693 tablet_copy_service.cc:140] P 58107b611f8f475a910a6fd54dfd1fb9: Received BeginTabletCopySession request for tablet a35103d97b874010b36c1863b2258c33 from peer da3f8160557e454f8547f45e80c2f013 ({username='slave'} at 127.26.25.136:47487)
I20240502 14:01:02.685098 693 tablet_copy_service.cc:161] P 58107b611f8f475a910a6fd54dfd1fb9: Beginning new tablet copy session on tablet a35103d97b874010b36c1863b2258c33 from peer da3f8160557e454f8547f45e80c2f013 at {username='slave'} at 127.26.25.136:47487: session id = da3f8160557e454f8547f45e80c2f013-a35103d97b874010b36c1863b2258c33
I20240502 14:01:02.685848 693 tablet_copy_source_session.cc:205] T a35103d97b874010b36c1863b2258c33 P 58107b611f8f475a910a6fd54dfd1fb9: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:02.686767 9722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35103d97b874010b36c1863b2258c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:02.691299 9722 tablet_copy_client.cc:783] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:02.691529 9722 tablet_copy_client.cc:647] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:02.692585 9722 tablet_copy_client.cc:523] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:02.695283 9722 tablet_bootstrap.cc:492] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013: Bootstrap starting.
I20240502 14:01:02.697346 9728 ts_tablet_manager.cc:902] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e: Initiating tablet copy from peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:01:02.697875 9728 tablet_copy_client.cc:308] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.134:39061
I20240502 14:01:02.698347 3039 tablet_copy_service.cc:140] P 452ecbc085e54db49c3b01c20a96f860: Received BeginTabletCopySession request for tablet 1eab750f98a6432494fa5351269e3052 from peer f003b732dd0f46958262b42f8d20305e ({username='slave'} at 127.26.25.129:35347)
I20240502 14:01:02.698459 3039 tablet_copy_service.cc:161] P 452ecbc085e54db49c3b01c20a96f860: Beginning new tablet copy session on tablet 1eab750f98a6432494fa5351269e3052 from peer f003b732dd0f46958262b42f8d20305e at {username='slave'} at 127.26.25.129:35347: session id = f003b732dd0f46958262b42f8d20305e-1eab750f98a6432494fa5351269e3052
I20240502 14:01:02.699570 3039 tablet_copy_source_session.cc:205] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:02.700568 9728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eab750f98a6432494fa5351269e3052. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:02.704566 9728 tablet_copy_client.cc:783] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:02.712011 9728 tablet_copy_client.cc:647] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:02.716051 9717 tablet_bootstrap.cc:492] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:02.716382 9717 tablet_bootstrap.cc:492] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5: Bootstrap complete.
I20240502 14:01:02.716576 9717 ts_tablet_manager.cc:1367] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5: Time spent bootstrapping tablet: real 0.036s user 0.020s sys 0.005s
I20240502 14:01:02.716645 9728 tablet_copy_client.cc:523] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:02.717034 9717 raft_consensus.cc:348] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:01:02.717157 9717 raft_consensus.cc:729] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Initialized, Role: LEARNER
I20240502 14:01:02.717268 9717 consensus_queue.cc:260] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { replace: true } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: NON_VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: true } }
I20240502 14:01:02.718586 9728 tablet_bootstrap.cc:492] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e: Bootstrap starting.
I20240502 14:01:02.719377 9717 ts_tablet_manager.cc:1398] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:01:02.719803 429 tablet_copy_service.cc:342] P f003b732dd0f46958262b42f8d20305e: Request end of tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-02a36ffbdd0f4d68953aa441c04b42eb received from {username='slave'} at 127.26.25.133:44833
I20240502 14:01:02.719908 429 tablet_copy_service.cc:433] P f003b732dd0f46958262b42f8d20305e: ending tablet copy session 4bcd7712aa844d98977a7d7e605c31f5-02a36ffbdd0f4d68953aa441c04b42eb on tablet 02a36ffbdd0f4d68953aa441c04b42eb with peer 4bcd7712aa844d98977a7d7e605c31f5
I20240502 14:01:02.723675 9728 tablet_bootstrap.cc:492] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:02.723966 9728 tablet_bootstrap.cc:492] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e: Bootstrap complete.
I20240502 14:01:02.724126 9728 ts_tablet_manager.cc:1367] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 14:01:02.724534 9728 raft_consensus.cc:348] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:01:02.724637 9728 raft_consensus.cc:729] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Initialized, Role: LEARNER
I20240502 14:01:02.724761 9728 consensus_queue.cc:260] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: NON_VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: true } }
I20240502 14:01:02.724898 9722 tablet_bootstrap.cc:492] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:02.725111 9728 ts_tablet_manager.cc:1398] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:02.725440 9722 tablet_bootstrap.cc:492] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013: Bootstrap complete.
I20240502 14:01:02.725602 9722 ts_tablet_manager.cc:1367] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013: Time spent bootstrapping tablet: real 0.030s user 0.025s sys 0.000s
I20240502 14:01:02.725688 3039 tablet_copy_service.cc:342] P 452ecbc085e54db49c3b01c20a96f860: Request end of tablet copy session f003b732dd0f46958262b42f8d20305e-1eab750f98a6432494fa5351269e3052 received from {username='slave'} at 127.26.25.129:35347
I20240502 14:01:02.725760 3039 tablet_copy_service.cc:433] P 452ecbc085e54db49c3b01c20a96f860: ending tablet copy session f003b732dd0f46958262b42f8d20305e-1eab750f98a6432494fa5351269e3052 on tablet 1eab750f98a6432494fa5351269e3052 with peer f003b732dd0f46958262b42f8d20305e
I20240502 14:01:02.726011 9722 raft_consensus.cc:348] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:01:02.726150 9722 raft_consensus.cc:729] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Initialized, Role: LEARNER
I20240502 14:01:02.727589 9722 consensus_queue.cc:260] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: NON_VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: true } }
I20240502 14:01:02.728034 9722 ts_tablet_manager.cc:1398] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:02.728701 693 tablet_copy_service.cc:342] P 58107b611f8f475a910a6fd54dfd1fb9: Request end of tablet copy session da3f8160557e454f8547f45e80c2f013-a35103d97b874010b36c1863b2258c33 received from {username='slave'} at 127.26.25.136:47487
I20240502 14:01:02.728789 693 tablet_copy_service.cc:433] P 58107b611f8f475a910a6fd54dfd1fb9: ending tablet copy session da3f8160557e454f8547f45e80c2f013-a35103d97b874010b36c1863b2258c33 on tablet a35103d97b874010b36c1863b2258c33 with peer da3f8160557e454f8547f45e80c2f013
I20240502 14:01:02.768752 9765 ts_tablet_manager.cc:902] T 1415dfd1daed4e1a8a00439c77185a55 P 2b6796cf0e2e496e9dd641beb432f485: Initiating tablet copy from peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
F20240502 14:01:02.768947 9765 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-00d3e2b37ce743bf90e211f5385efe90.queue_time_us, tablet-download-pool-00d3e2b37ce743bf90e211f5385efe90.queue_time_us) (tablet-download-pool-00d3e2b37ce743bf90e211f5385efe90.run_cpu_time_us, tablet-download-pool-00d3e2b37ce743bf90e211f5385efe90.run_cpu_time_us) (tablet-download-pool-00d3e2b37ce743bf90e211f5385efe90.run_wall_time_us, tablet-download-pool-00d3e2b37ce743bf90e211f5385efe90.run_wall_time_us) (tablet-download-pool-10d6cc99575f4ebb8c980e9cb3e35145.queue_time_us, tablet-download-pool-10d6cc99575f4ebb8c980e9cb3e35145.queue_time_us) (tablet-download-pool-10d6cc99575f4ebb8c980e9cb3e35145.run_cpu_time_us, tablet-download-pool-10d6cc99575f4ebb8c980e9cb3e35145.run_cpu_time_us) (tablet-download-pool-10d6cc99575f4ebb8c980e9cb3e35145.run_wall_time_us, tablet-download-pool-10d6cc99575f4ebb8c980e9cb3e35145.run_wall_time_us) (tablet-download-pool-1415dfd1daed4e1a8a00439c77185a55.queue_time_us, tablet-download-pool-1415dfd1daed4e1a8a00439c77185a55.queue_time_us) (tablet-download-pool-3f9b1351891d4690b67f8eef000a902b.queue_time_us, tablet-download-pool-3f9b1351891d4690b67f8eef000a902b.queue_time_us) (tablet-download-pool-3f9b1351891d4690b67f8eef000a902b.run_cpu_time_us, tablet-download-pool-3f9b1351891d4690b67f8eef000a902b.run_cpu_time_us) (tablet-download-pool-3f9b1351891d4690b67f8eef000a902b.run_wall_time_us, tablet-download-pool-3f9b1351891d4690b67f8eef000a902b.run_wall_time_us) (tablet-download-pool-59dd5ee0b3554ac493ce2d4d15a99abd.queue_time_us, tablet-download-pool-59dd5ee0b3554ac493ce2d4d15a99abd.queue_time_us) (tablet-download-pool-59dd5ee0b3554ac493ce2d4d15a99abd.run_cpu_time_us, tablet-download-pool-59dd5ee0b3554ac493ce2d4d15a99abd.run_cpu_time_us) (tablet-download-pool-59dd5ee0b3554ac493ce2d4d15a99abd.run_wall_time_us, tablet-download-pool-59dd5ee0b3554ac493ce2d4d15a99abd.run_wall_time_us) (tablet-download-pool-5f321621da5d481b8d1d2b25ddf4b395.queue_time_us, tablet-download-pool-5f321621da5d481b8d1d2b25ddf4b395.queue_time_us) (tablet-download-pool-5f321621da5d481b8d1d2b25ddf4b395.run_cpu_time_us, tablet-download-pool-5f321621da5d481b8d1d2b25ddf4b395.run_cpu_time_us) (tablet-download-pool-5f321621da5d481b8d1d2b25ddf4b395.run_wall_time_us, tablet-download-pool-5f321621da5d481b8d1d2b25ddf4b395.run_wall_time_us) (tablet-download-pool-7113a6da4229422099dbd604a560fb95.queue_time_us, tablet-download-pool-7113a6da4229422099dbd604a560fb95.queue_time_us) (tablet-download-pool-7113a6da4229422099dbd604a560fb95.run_cpu_time_us, tablet-download-pool-7113a6da4229422099dbd604a560fb95.run_cpu_time_us) (tablet-download-pool-7113a6da4229422099dbd604a560fb95.run_wall_time_us, tablet-download-pool-7113a6da4229422099dbd604a560fb95.run_wall_time_us) (tablet-download-pool-8568aa57fce64ccf89a36deb5b7b0f3b.queue_time_us, tablet-download-pool-8568aa57fce64ccf89a36deb5b7b0f3b.queue_time_us) (tablet-download-pool-8568aa57fce64ccf89a36deb5b7b0f3b.run_cpu_time_us, tablet-download-pool-8568aa57fce64ccf89a36deb5b7b0f3b.run_cpu_time_us) (tablet-download-pool-8568aa57fce64ccf89a36deb5b7b0f3b.run_wall_time_us, tablet-download-pool-8568aa57fce64ccf89a36deb5b7b0f3b.run_wall_time_us) (tablet-download-pool-9cd00d821b5d4b35ada028a8c83d0ed7.queue_time_us, tablet-download-pool-9cd00d821b5d4b35ada028a8c83d0ed7.queue_time_us) (tablet-download-pool-9cd00d821b5d4b35ada028a8c83d0ed7.run_cpu_time_us, tablet-download-pool-9cd00d821b5d4b35ada028a8c83d0ed7.run_cpu_time_us) (tablet-download-pool-9cd00d821b5d4b35ada028a8c83d0ed7.run_wall_time_us, tablet-download-pool-9cd00d821b5d4b35ada028a8c83d0ed7.run_wall_time_us) (tablet-download-pool-c47fb9f1470349b3be140e7b70811e7b.queue_time_us, tablet-download-pool-c47fb9f1470349b3be140e7b70811e7b.queue_time_us) (tablet-download-pool-c47fb9f1470349b3be140e7b70811e7b.run_cpu_time_us, tablet-download-pool-c47fb9f1470349b3be140e7b70811e7b.run_cpu_time_us) (tablet-download-pool-c47fb9f1470349b3be140e7b70811e7b.run_wall_time_us, tablet-download-pool-c47fb9f1470349b3be140e7b70811e7b.run_wall_time_us) (tablet-download-pool-c8e843cd7c984ebfb532fec67f97e249.queue_time_us, tablet-download-pool-c8e843cd7c984ebfb532fec67f97e249.queue_time_us) (tablet-download-pool-c8e843cd7c984ebfb532fec67f97e249.run_cpu_time_us, tablet-download-pool-c8e843cd7c984ebfb532fec67f97e249.run_cpu_time_us) (tablet-download-pool-c8e843cd7c984ebfb532fec67f97e249.run_wall_time_us, tablet-download-pool-c8e843cd7c984ebfb532fec67f97e249.run_wall_time_us) (tablet-download-pool-caffa941c89c48d1b3d36ca47dc49eb2.queue_time_us, tablet-download-pool-caffa941c89c48d1b3d36ca47dc49eb2.queue_time_us) (tablet-download-pool-caffa941c89c48d1b3d36ca47dc49eb2.run_cpu_time_us, tablet-download-pool-caffa941c89c48d1b3d36ca47dc49eb2.run_cpu_time_us) (tablet-download-pool-caffa941c89c48d1b3d36ca47dc49eb2.run_wall_time_us, tablet-download-pool-caffa941c89c48d1b3d36ca47dc49eb2.run_wall_time_us) (tablet-download-pool-ce81659e36874bb89da4d2268eb40aea.queue_time_us, tablet-download-pool-ce81659e36874bb89da4d2268eb40aea.queue_time_us) (tablet-download-pool-ce81659e36874bb89da4d2268eb40aea.run_cpu_time_us, tablet-download-pool-ce81659e36874bb89da4d2268eb40aea.run_cpu_time_us) (tablet-download-pool-ce81659e36874bb89da4d2268eb40aea.run_wall_time_us, tablet-download-pool-ce81659e36874bb89da4d2268eb40aea.run_wall_time_us) (tablet-download-pool-d493cec70a39401b98c21446f866789a.queue_time_us, tablet-download-pool-d493cec70a39401b98c21446f866789a.queue_time_us) (tablet-download-pool-d493cec70a39401b98c21446f866789a.run_cpu_time_us, tablet-download-pool-d493cec70a39401b98c21446f866789a.run_cpu_time_us) (tablet-download-pool-d493cec70a39401b98c21446f866789a.run_wall_time_us, tablet-download-pool-d493cec70a39401b98c21446f866789a.run_wall_time_us) (tablet-download-pool-fee3c05d88514648b8291e01eda2db2b.queue_time_us, tablet-download-pool-fee3c05d88514648b8291e01eda2db2b.queue_time_us) (tablet-download-pool-fee3c05d88514648b8291e01eda2db2b.run_cpu_time_us, tablet-download-pool-fee3c05d88514648b8291e01eda2db2b.run_cpu_time_us) (tablet-download-pool-fee3c05d88514648b8291e01eda2db2b.run_wall_time_us, tablet-download-pool-fee3c05d88514648b8291e01eda2db2b.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
@ 0x7f4825d49dcd google::LogMessage::Fail() at ??:0
@ 0x7f4825d4db93 google::LogMessage::SendToLog() at ??:0
@ 0x7f4825d497cc google::LogMessage::Flush() at ??:0
@ 0x7f4825d4af59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f48271b3044 kudu::TraceMetrics::InternName() at ??:0
@ 0x7f4827197f98 kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7f4827195efc kudu::ThreadPoolBuilder::Build() at ??:0
I20240502 14:01:02.793534 9704 raft_consensus.cc:1050] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER 2b6796cf0e2e496e9dd641beb432f485 to VOTER
I20240502 14:01:02.794066 9704 consensus_queue.cc:237] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:01:02.795574 3284 raft_consensus.cc:1261] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:02.796253 3150 raft_consensus.cc:1261] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:02.796662 9704 consensus_queue.cc:1035] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
@ 0x7f4829d4877a kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
I20240502 14:01:02.797173 3019 raft_consensus.cc:1261] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
@ 0x7f4829d51a45 kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
I20240502 14:01:02.800431 7206 consensus_queue.cc:1035] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:02.801149 7206 consensus_queue.cc:1035] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:02.803298 7206 raft_consensus.cc:482] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:02.803380 7206 raft_consensus.cc:504] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:01:02.803802 7206 leader_election.cc:288] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:02.803768 8167 raft_consensus.cc:2932] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:01:02.804272 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78ea0e379fa4f9786c7b2e45c9c8cc9" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:01:02.805249 3150 raft_consensus.cc:2932] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
@ 0x7f4829dd276f kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
I20240502 14:01:02.808475 32732 catalog_manager.cc:5484] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:02.809337 3283 raft_consensus.cc:2932] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
@ 0x7f4829de64f6 kudu::tserver::TabletCopyRunnable::Run() at ??:0
W20240502 14:01:02.813210 2822 leader_election.cc:334] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.813282 2822 leader_election.cc:302] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5; no voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751
I20240502 14:01:02.813591 7206 raft_consensus.cc:2726] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 14:01:02.814651 2885 consensus_queue.cc:237] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } }
I20240502 14:01:02.816406 3283 raft_consensus.cc:1261] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:02.816659 3152 raft_consensus.cc:1261] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:02.818117 9704 consensus_queue.cc:1035] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:02.818217 7206 consensus_queue.cc:1035] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
@ 0x7f4829dd1226 _ZZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS_6StatusENS0_24TabletServerErrorPB_CodeEEEENKUlvE_clEv at ??:0
I20240502 14:01:02.821875 8358 raft_consensus.cc:2932] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:01:02.822556 3152 raft_consensus.cc:2932] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
I20240502 14:01:02.823137 3283 raft_consensus.cc:2932] T d493cec70a39401b98c21446f866789a P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
@ 0x7f4829de0c0a _ZNSt17_Function_handlerIFvvEZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS1_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS1_6StatusENS2_24TabletServerErrorPB_CodeEEEEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x5588e66ab75c std::function<>::operator()() at ??:0
I20240502 14:01:02.825274 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d493cec70a39401b98c21446f866789a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:02.827665 32732 catalog_manager.cc:5484] T d493cec70a39401b98c21446f866789a P 2b6796cf0e2e496e9dd641beb432f485 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } }
@ 0x7f482719ae06 kudu::ThreadPool::DispatchThread() at ??:0
I20240502 14:01:02.830914 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet d493cec70a39401b98c21446f866789a with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
@ 0x7f482719b6ef _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f482719d03b _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x5588e66ab75c std::function<>::operator()() at ??:0
I20240502 14:01:02.839113 9795 tablet_replica.cc:330] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:01:02.839284 9795 raft_consensus.cc:2229] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:02.839417 9795 pending_rounds.cc:62] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860: Trying to abort 1 pending ops.
I20240502 14:01:02.839449 9795 pending_rounds.cc:68] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7023241063603154944 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "d493cec70a39401b98c21446f866789a" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: NON_VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { promote: false } } } }
I20240502 14:01:02.839568 9795 raft_consensus.cc:2894] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20240502 14:01:02.839612 9795 raft_consensus.cc:2866] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 14:01:02.839641 9795 raft_consensus.cc:2258] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7f482718a807 kudu::Thread::SuperviseThread() at ??:0
I20240502 14:01:02.840219 9795 ts_tablet_manager.cc:1875] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
@ 0x7f48279c86db start_thread at ??:0
I20240502 14:01:02.842166 9795 ts_tablet_manager.cc:1888] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:02.842254 9795 log.cc:1196] T d493cec70a39401b98c21446f866789a P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/d493cec70a39401b98c21446f866789a
I20240502 14:01:02.842861 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet d493cec70a39401b98c21446f866789a (table rebalancer_extra_table_48 [id=ad5b5fff56c246e3868d58f2a1533bac]) successfully deleted
I20240502 14:01:02.845746 9320 raft_consensus.cc:482] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
@ 0x7f4824f0071f clone at ??:0
I20240502 14:01:02.845856 9320 raft_consensus.cc:504] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:02.846325 9320 leader_election.cc:288] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:02.846837 673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a192eba0af4486a212972ffdfacf9d" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
W20240502 14:01:02.847528 477 leader_election.cc:334] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.847607 477 leader_election.cc:302] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751; no voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:01:02.847895 9320 raft_consensus.cc:2726] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 14:01:02.851166 9707 ts_tablet_manager.cc:902] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9: Initiating tablet copy from peer 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:01:02.851665 9707 tablet_copy_client.cc:308] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Beginning tablet copy session from remote peer at address 127.26.25.134:39061
I20240502 14:01:02.852519 3039 tablet_copy_service.cc:140] P 452ecbc085e54db49c3b01c20a96f860: Received BeginTabletCopySession request for tablet 844d8895be4b4635bc21650b60049409 from peer 58107b611f8f475a910a6fd54dfd1fb9 ({username='slave'} at 127.26.25.131:44533)
I20240502 14:01:02.852600 3039 tablet_copy_service.cc:161] P 452ecbc085e54db49c3b01c20a96f860: Beginning new tablet copy session on tablet 844d8895be4b4635bc21650b60049409 from peer 58107b611f8f475a910a6fd54dfd1fb9 at {username='slave'} at 127.26.25.131:44533: session id = 58107b611f8f475a910a6fd54dfd1fb9-844d8895be4b4635bc21650b60049409
I20240502 14:01:02.853261 3039 tablet_copy_source_session.cc:205] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:02.854158 9707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 844d8895be4b4635bc21650b60049409. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:02.857141 9707 tablet_copy_client.cc:783] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:02.857300 9707 tablet_copy_client.cc:647] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:02.857677 8205 raft_consensus.cc:482] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:02.857784 8205 raft_consensus.cc:504] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:01:02.858247 8205 leader_election.cc:288] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:02.858369 9707 tablet_copy_client.cc:523] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:02.858784 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9486274cd39b4de18c37ab9f71f261c2" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
W20240502 14:01:02.859444 3219 leader_election.cc:334] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.859505 3219 leader_election.cc:302] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3f8160557e454f8547f45e80c2f013; no voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e
I20240502 14:01:02.859722 8205 raft_consensus.cc:2726] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 14:01:02.860843 9707 tablet_bootstrap.cc:492] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap starting.
I20240502 14:01:02.865489 9707 tablet_bootstrap.cc:492] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:02.865731 9707 tablet_bootstrap.cc:492] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9: Bootstrap complete.
I20240502 14:01:02.865875 9707 ts_tablet_manager.cc:1367] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:01:02.866508 9707 raft_consensus.cc:348] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:02.866619 9707 raft_consensus.cc:729] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Initialized, Role: LEARNER
I20240502 14:01:02.866714 9707 consensus_queue.cc:260] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } attrs { replace: true } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: NON_VOTER last_known_addr { host: "127.26.25.131" port: 38887 } attrs { promote: true } }
I20240502 14:01:02.867089 9707 ts_tablet_manager.cc:1398] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:01:02.868757 3039 tablet_copy_service.cc:342] P 452ecbc085e54db49c3b01c20a96f860: Request end of tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-844d8895be4b4635bc21650b60049409 received from {username='slave'} at 127.26.25.131:44533
I20240502 14:01:02.868834 3039 tablet_copy_service.cc:433] P 452ecbc085e54db49c3b01c20a96f860: ending tablet copy session 58107b611f8f475a910a6fd54dfd1fb9-844d8895be4b4635bc21650b60049409 on tablet 844d8895be4b4635bc21650b60049409 with peer 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:01:02.900432 9320 raft_consensus.cc:482] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:02.900560 9320 raft_consensus.cc:504] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:01:02.901041 9320 leader_election.cc:288] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:02.901623 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78ea0e379fa4f9786c7b2e45c9c8cc9" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:01:02.901747 2885 raft_consensus.cc:2454] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 1.
I20240502 14:01:02.902276 477 leader_election.cc:302] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters:
W20240502 14:01:02.902412 477 leader_election.cc:334] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.902453 9320 raft_consensus.cc:2781] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 14:01:02.902494 9320 raft_consensus.cc:482] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:02.902530 9320 raft_consensus.cc:3037] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:02.903496 9320 raft_consensus.cc:504] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:01:02.903781 9320 leader_election.cc:288] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 election: Requested vote from peers 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:02.904165 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78ea0e379fa4f9786c7b2e45c9c8cc9" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5"
I20240502 14:01:02.904291 2885 raft_consensus.cc:3037] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:01:02.904852 477 leader_election.cc:334] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.905478 2885 raft_consensus.cc:2454] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 2.
I20240502 14:01:02.905839 477 leader_election.cc:302] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters: 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:02.905961 9393 raft_consensus.cc:482] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:02.906042 9320 raft_consensus.cc:2781] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:02.906071 9393 raft_consensus.cc:504] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:01:02.906136 9320 raft_consensus.cc:686] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [term 2 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:01:02.906284 9320 consensus_queue.cc:237] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:01:02.906508 9393 leader_election.cc:288] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:01:02.907263 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
W20240502 14:01:02.908844 2954 leader_election.cc:334] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.908913 2954 leader_election.cc:302] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860; no voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e
I20240502 14:01:02.909106 9393 raft_consensus.cc:2726] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20240502 14:01:02.911005 32729 catalog_manager.cc:5484] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 1 to 2, leader changed from 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 2 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:02.957630 8205 raft_consensus.cc:482] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:02.957780 8205 raft_consensus.cc:504] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:01:02.958230 8205 leader_election.cc:288] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:02.958626 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
W20240502 14:01:02.959511 3219 leader_election.cc:334] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.959590 3219 leader_election.cc:302] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3f8160557e454f8547f45e80c2f013; no voters: 478dfa53d7944f38b4e6cabee21b2d01, c351a89a5fe34e788b9d8064fbffa751
I20240502 14:01:02.959728 8205 raft_consensus.cc:2726] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20240502 14:01:02.995370 9713 raft_consensus.cc:482] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:02.995507 9713 raft_consensus.cc:504] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:01:02.995966 9713 leader_election.cc:288] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:01:02.996680 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:01:02.996812 3283 raft_consensus.cc:2454] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 2.
I20240502 14:01:02.997120 475 leader_election.cc:302] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters:
W20240502 14:01:02.997277 477 leader_election.cc:334] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:02.997290 9713 raft_consensus.cc:2781] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20240502 14:01:02.997494 9713 raft_consensus.cc:482] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:02.997538 9713 raft_consensus.cc:3037] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:02.998701 9713 raft_consensus.cc:504] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:01:02.999045 9713 leader_election.cc:288] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 3 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:01:02.999991 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5bed83e31be4129a19eb8c347514b80" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:01:03.000108 3283 raft_consensus.cc:3037] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:03.001165 3283 raft_consensus.cc:2454] T e5bed83e31be4129a19eb8c347514b80 P da3f8160557e454f8547f45e80c2f013 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c351a89a5fe34e788b9d8064fbffa751 in term 3.
I20240502 14:01:03.001539 475 leader_election.cc:302] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters:
W20240502 14:01:03.002281 477 leader_election.cc:334] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.003505 9713 raft_consensus.cc:2781] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:03.003600 9713 raft_consensus.cc:686] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [term 3 LEADER]: Becoming Leader. State: Replica: c351a89a5fe34e788b9d8064fbffa751, State: Running, Role: LEADER
I20240502 14:01:03.003731 9713 consensus_queue.cc:237] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:01:03.008477 32729 catalog_manager.cc:5484] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 reported cstate change: term changed from 2 to 3, leader changed from 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) to c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130). New cstate: current_term: 3 leader_uuid: "c351a89a5fe34e788b9d8064fbffa751" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:03.008673 9592 raft_consensus.cc:482] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.008805 9592 raft_consensus.cc:504] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:01:03.009200 9592 leader_election.cc:288] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:01:03.009536 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9486274cd39b4de18c37ab9f71f261c2" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
I20240502 14:01:03.009658 3283 raft_consensus.cc:2454] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:01:03.010025 342 leader_election.cc:302] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:01:03.011487 9592 raft_consensus.cc:2781] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 14:01:03.011557 9592 raft_consensus.cc:482] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.011597 9592 raft_consensus.cc:3037] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:01:03.011902 344 leader_election.cc:334] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.012850 9592 raft_consensus.cc:504] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:01:03.013236 9592 leader_election.cc:288] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131)
I20240502 14:01:03.013546 3283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9486274cd39b4de18c37ab9f71f261c2" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013"
I20240502 14:01:03.013657 3283 raft_consensus.cc:3037] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:01:03.014510 344 leader_election.cc:334] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.014878 3283 raft_consensus.cc:2454] T 9486274cd39b4de18c37ab9f71f261c2 P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 2.
I20240502 14:01:03.015161 342 leader_election.cc:302] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: da3f8160557e454f8547f45e80c2f013, f003b732dd0f46958262b42f8d20305e; no voters: 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.015350 9592 raft_consensus.cc:2781] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:03.015437 9592 raft_consensus.cc:686] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [term 2 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:01:03.015575 9592 consensus_queue.cc:237] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:01:03.018030 32729 catalog_manager.cc:5484] T 9486274cd39b4de18c37ab9f71f261c2 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 1 to 2, leader changed from 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 2 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:03.025036 9713 raft_consensus.cc:482] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.025141 9713 raft_consensus.cc:504] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:01:03.025496 9713 leader_election.cc:288] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:03.025784 673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c3caa06e284ef2ad50c6badfa8f59e" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
W20240502 14:01:03.026595 477 leader_election.cc:334] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.026665 477 leader_election.cc:302] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751; no voters: 478dfa53d7944f38b4e6cabee21b2d01, 58107b611f8f475a910a6fd54dfd1fb9
I20240502 14:01:03.026842 9713 raft_consensus.cc:2726] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 14:01:03.036125 609 connection.cc:535] client connection to 127.26.25.135:45509 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:03.036952 2822 connection.cc:535] client connection to 127.26.25.135:45509 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:03.038075 2954 connection.cc:535] client connection to 127.26.25.135:45509 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:03.038408 477 connection.cc:535] client connection to 127.26.25.135:45509 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:03.038982 9502 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 14:01:03.037395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.25.135:45509 (local address 127.26.25.133:54767)
0502 14:01:03.037548 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0502 14:01:03.037561 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0502 14:01:03.037647 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 14:01:03.038380 (+ 733us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":91}
W20240502 14:01:03.039126 2822 consensus_peers.cc:485] T 00d3e2b37ce743bf90e211f5385efe90 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.039119 3219 connection.cc:535] client connection to 127.26.25.135:45509 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 14:01:03.039178 2822 consensus_peers.cc:485] T 2fcc29071c964817ac2b517c563027ab P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.040019 477 consensus_peers.cc:485] T db0bf9f6821045e1a259703fc3445226 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.040102 477 consensus_peers.cc:485] T fee3c05d88514648b8291e01eda2db2b P c351a89a5fe34e788b9d8064fbffa751 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.040154 477 consensus_peers.cc:462] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111)
W20240502 14:01:03.040266 2954 consensus_peers.cc:485] T 6451a34c3fb34207bcd761556b0e08dd P 452ecbc085e54db49c3b01c20a96f860 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.040345 2954 consensus_peers.cc:485] T 413dd188108549d6b19b7d13bbfc3779 P 452ecbc085e54db49c3b01c20a96f860 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.040388 2954 consensus_peers.cc:485] T c47fb9f1470349b3be140e7b70811e7b P 452ecbc085e54db49c3b01c20a96f860 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.040813 9505 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 14:01:03.037884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.25.135:45509 (local address 127.26.25.131:37863)
0502 14:01:03.038904 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0502 14:01:03.038917 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0502 14:01:03.039008 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 14:01:03.040392 (+ 1384us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":952}
W20240502 14:01:03.040931 609 consensus_peers.cc:485] T 5f321621da5d481b8d1d2b25ddf4b395 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.041004 609 consensus_peers.cc:485] T c8e843cd7c984ebfb532fec67f97e249 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.041781 3219 consensus_peers.cc:485] T e750578ccc314e1eaa17b25873cdb3cf P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.041862 3219 consensus_peers.cc:485] T 9cd00d821b5d4b35ada028a8c83d0ed7 P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.041915 3219 consensus_peers.cc:485] T 923a5d2ccd0448b381d88ced69ae8fe2 P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.041965 3219 consensus_peers.cc:485] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.042017 3219 consensus_peers.cc:485] T 9f6b40db906d4d72bae26a007737789b P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.044519 9583 raft_consensus.cc:1050] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9: attempting to promote NON_VOTER 4bcd7712aa844d98977a7d7e605c31f5 to VOTER
I20240502 14:01:03.044909 9583 consensus_queue.cc:237] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:01:03.046504 541 raft_consensus.cc:1261] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:03.048017 9318 consensus_queue.cc:1035] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240502 14:01:03.048650 3219 consensus_peers.cc:485] T 2423568c1fef4f118afa96ad22747159 P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.049475 2885 raft_consensus.cc:1261] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Refusing update from remote peer 58107b611f8f475a910a6fd54dfd1fb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:03.049549 6065 raft_consensus.cc:482] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.049635 6065 raft_consensus.cc:504] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:01:03.049965 6065 leader_election.cc:288] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:01:03.050549 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9fc8aa46794f459538e925564e0693" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:01:03.051587 6065 consensus_queue.cc:1035] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240502 14:01:03.051908 609 leader_election.cc:334] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.051980 609 leader_election.cc:302] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9; no voters: 478dfa53d7944f38b4e6cabee21b2d01, f003b732dd0f46958262b42f8d20305e
I20240502 14:01:03.052109 6065 raft_consensus.cc:2726] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 14:01:03.052142 609 consensus_peers.cc:485] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.055725 3219 consensus_peers.cc:485] T 907ccfaebc3c4181bda0d3f03aba050c P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.059028 2822 consensus_peers.cc:485] T f3d9ae2ed9a84c2c928dc1d03f325fa3 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.059957 9318 raft_consensus.cc:2932] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:03.060473 541 raft_consensus.cc:2932] T 8689975c1d9d41f4ae7152bf6e4a58a8 P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:03.062438 2885 raft_consensus.cc:2932] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } } }
I20240502 14:01:03.063555 32729 catalog_manager.cc:5484] T 8689975c1d9d41f4ae7152bf6e4a58a8 P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: config changed from index 4 to 5, 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:03.064024 9592 raft_consensus.cc:482] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.064122 9592 raft_consensus.cc:504] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:01:03.064515 9592 leader_election.cc:288] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061)
I20240502 14:01:03.065276 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad896b8838472096851682071c0a9c" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:01:03.065786 7206 raft_consensus.cc:1050] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5: attempting to promote NON_VOTER c351a89a5fe34e788b9d8064fbffa751 to VOTER
I20240502 14:01:03.066175 7206 consensus_queue.cc:237] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
W20240502 14:01:03.067567 344 leader_election.cc:334] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.067636 344 leader_election.cc:302] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f003b732dd0f46958262b42f8d20305e; no voters: 452ecbc085e54db49c3b01c20a96f860, 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.067915 9592 raft_consensus.cc:2726] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 14:01:03.068609 3219 consensus_peers.cc:485] T 81cd2e05295641678fa8d7711c797113 P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.069537 541 raft_consensus.cc:1261] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751 [term 1 LEARNER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:03.069561 3019 raft_consensus.cc:1261] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:03.070227 3283 raft_consensus.cc:1261] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:03.070983 9704 consensus_queue.cc:1035] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:03.071205 8167 consensus_queue.cc:1035] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:03.071375 7206 consensus_queue.cc:1035] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:01:03.075107 8167 raft_consensus.cc:2932] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:03.075593 541 raft_consensus.cc:2932] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:03.076205 3019 raft_consensus.cc:2932] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:03.078857 32732 catalog_manager.cc:5484] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:03.080762 3284 raft_consensus.cc:2932] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { replace: true } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:03.082509 2885 consensus_queue.cc:237] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:03.083781 3283 raft_consensus.cc:1261] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:03.084558 9704 consensus_queue.cc:1035] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:01:03.085621 3219 consensus_peers.cc:485] T ed1ae34edc6b4b6c864a6b7659810475 P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.087837 541 raft_consensus.cc:1261] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd7712aa844d98977a7d7e605c31f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:03.087956 9704 raft_consensus.cc:2932] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:03.088173 3284 raft_consensus.cc:2932] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:03.088339 7206 consensus_queue.cc:1035] T 0dc1880244a64b9f96ad5b8122a49fce P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:03.089303 541 raft_consensus.cc:2932] T 0dc1880244a64b9f96ad5b8122a49fce P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:03.089929 32721 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0dc1880244a64b9f96ad5b8122a49fce with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:03.092621 32726 catalog_manager.cc:5484] T 0dc1880244a64b9f96ad5b8122a49fce P da3f8160557e454f8547f45e80c2f013 reported cstate change: config changed from index 3 to 4, VOTER 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134) evicted. New cstate: current_term: 1 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } } }
I20240502 14:01:03.095381 2999 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dc1880244a64b9f96ad5b8122a49fce with delete_type TABLET_DATA_TOMBSTONED (TS 452ecbc085e54db49c3b01c20a96f860 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57722
I20240502 14:01:03.095597 9795 tablet_replica.cc:330] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860: stopping tablet replica
I20240502 14:01:03.095765 9795 raft_consensus.cc:2229] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:03.095944 9795 raft_consensus.cc:2258] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:03.096860 9795 ts_tablet_manager.cc:1875] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:03.097222 6065 raft_consensus.cc:482] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.097309 6065 raft_consensus.cc:504] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:01:03.097638 6065 leader_election.cc:288] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
W20240502 14:01:03.098326 609 leader_election.cc:334] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.099009 9795 ts_tablet_manager.cc:1888] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:01:03.099078 9795 log.cc:1196] T 0dc1880244a64b9f96ad5b8122a49fce P 452ecbc085e54db49c3b01c20a96f860: Deleting WAL directory at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/wals/0dc1880244a64b9f96ad5b8122a49fce
I20240502 14:01:03.099561 32718 catalog_manager.cc:4830] TS 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061): tablet 0dc1880244a64b9f96ad5b8122a49fce (table rebalancer_extra_table_6 [id=5b4333df1acc4bd18638ef4d477138b9]) successfully deleted
I20240502 14:01:03.099722 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c3caa06e284ef2ad50c6badfa8f59e" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:01:03.099848 541 raft_consensus.cc:2454] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:01:03.100118 606 leader_election.cc:302] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters: 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.100389 6065 raft_consensus.cc:2781] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 14:01:03.100454 6065 raft_consensus.cc:482] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.100512 6065 raft_consensus.cc:3037] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:03.101418 6065 raft_consensus.cc:504] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:01:03.101734 6065 leader_election.cc:288] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 election: Requested vote from peers c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:03.102463 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2c3caa06e284ef2ad50c6badfa8f59e" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:01:03.102581 541 raft_consensus.cc:3037] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:01:03.103446 344 consensus_peers.cc:485] T a176c4678de94ccc93b14d04fc46ea90 P f003b732dd0f46958262b42f8d20305e -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.103648 541 raft_consensus.cc:2454] T a2c3caa06e284ef2ad50c6badfa8f59e P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 2.
W20240502 14:01:03.103775 477 consensus_peers.cc:485] T 8568aa57fce64ccf89a36deb5b7b0f3b P c351a89a5fe34e788b9d8064fbffa751 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.103971 606 leader_election.cc:302] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:01:03.104353 6065 raft_consensus.cc:2781] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:03.104439 6065 raft_consensus.cc:686] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
W20240502 14:01:03.104508 609 leader_election.cc:334] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
W20240502 14:01:03.104915 477 consensus_peers.cc:485] T 7d23725b6cff48cbbf546394a4c54406 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.105085 6065 consensus_queue.cc:237] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:01:03.107996 2885 raft_consensus.cc:1203] T 02a36ffbdd0f4d68953aa441c04b42eb P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:03.108027 32726 catalog_manager.cc:5484] T a2c3caa06e284ef2ad50c6badfa8f59e P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 1 to 2, leader changed from 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:03.113317 344 consensus_peers.cc:485] T c1dc8816aa514d1abcce63ce0e0bf357 P f003b732dd0f46958262b42f8d20305e -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.114558 3219 consensus_peers.cc:485] T 7eb2f6c66b714668ab1b698fa4f6cc6c P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.115113 9592 raft_consensus.cc:482] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.115204 9592 raft_consensus.cc:504] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:01:03.115559 9592 leader_election.cc:288] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:03.115798 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9fc8aa46794f459538e925564e0693" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9" is_pre_election: true
I20240502 14:01:03.115976 672 raft_consensus.cc:2454] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 1.
I20240502 14:01:03.116240 341 leader_election.cc:302] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:01:03.116436 9592 raft_consensus.cc:2781] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 14:01:03.116513 9592 raft_consensus.cc:482] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.116926 9592 raft_consensus.cc:3037] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:03.117978 9592 raft_consensus.cc:504] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:01:03.118314 9592 leader_election.cc:288] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 election: Requested vote from peers 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:03.118718 672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9fc8aa46794f459538e925564e0693" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "58107b611f8f475a910a6fd54dfd1fb9"
I20240502 14:01:03.118832 672 raft_consensus.cc:3037] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:01:03.118845 344 leader_election.cc:334] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
W20240502 14:01:03.119463 344 leader_election.cc:334] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.120148 672 raft_consensus.cc:2454] T 3c9fc8aa46794f459538e925564e0693 P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 2.
I20240502 14:01:03.120630 341 leader_election.cc:302] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, f003b732dd0f46958262b42f8d20305e; no voters: 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.121165 9592 raft_consensus.cc:2781] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:03.121250 9592 raft_consensus.cc:686] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [term 2 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:01:03.121390 9592 consensus_queue.cc:237] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } }
I20240502 14:01:03.123857 32729 catalog_manager.cc:5484] T 3c9fc8aa46794f459538e925564e0693 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 1 to 2, leader changed from 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 2 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: HEALTHY } } }
W20240502 14:01:03.157486 477 consensus_peers.cc:462] T 1415dfd1daed4e1a8a00439c77185a55 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111)
W20240502 14:01:03.164184 609 consensus_peers.cc:485] T 7113a6da4229422099dbd604a560fb95 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.168630 9854 raft_consensus.cc:482] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.168756 9854 raft_consensus.cc:504] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:01:03.169181 9854 leader_election.cc:288] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:01:03.169449 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad896b8838472096851682071c0a9c" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e" is_pre_election: true
I20240502 14:01:03.169577 409 raft_consensus.cc:2454] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 1.
I20240502 14:01:03.169838 2953 leader_election.cc:302] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:01:03.169998 9854 raft_consensus.cc:2781] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 14:01:03.170053 9854 raft_consensus.cc:482] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.170094 9854 raft_consensus.cc:3037] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:01:03.170166 2954 leader_election.cc:334] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.171227 9854 raft_consensus.cc:504] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:01:03.171556 9854 leader_election.cc:288] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715)
I20240502 14:01:03.172029 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad896b8838472096851682071c0a9c" candidate_uuid: "452ecbc085e54db49c3b01c20a96f860" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: false dest_uuid: "f003b732dd0f46958262b42f8d20305e"
I20240502 14:01:03.172147 409 raft_consensus.cc:3037] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:01:03.172533 2954 leader_election.cc:334] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.173238 409 raft_consensus.cc:2454] T 85ad896b8838472096851682071c0a9c P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 452ecbc085e54db49c3b01c20a96f860 in term 2.
W20240502 14:01:03.173346 3219 consensus_peers.cc:462] T c80435eecda64ecea7864522f5a1dd4c P da3f8160557e454f8547f45e80c2f013 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.173979 2953 leader_election.cc:302] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters: 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.174202 9854 raft_consensus.cc:2781] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:03.174289 9854 raft_consensus.cc:686] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [term 2 LEADER]: Becoming Leader. State: Replica: 452ecbc085e54db49c3b01c20a96f860, State: Running, Role: LEADER
I20240502 14:01:03.174423 9854 consensus_queue.cc:237] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } }
I20240502 14:01:03.178027 32732 catalog_manager.cc:5484] T 85ad896b8838472096851682071c0a9c P 452ecbc085e54db49c3b01c20a96f860 reported cstate change: term changed from 1 to 2, leader changed from 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) to 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134). New cstate: current_term: 2 leader_uuid: "452ecbc085e54db49c3b01c20a96f860" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:03.192483 9582 raft_consensus.cc:482] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.192582 9582 raft_consensus.cc:504] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:03.192940 9582 leader_election.cc:288] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:01:03.193378 3284 raft_consensus.cc:1203] T a35103d97b874010b36c1863b2258c33 P da3f8160557e454f8547f45e80c2f013 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:03.194296 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a192eba0af4486a212972ffdfacf9d" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:01:03.194425 541 raft_consensus.cc:2454] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 1.
I20240502 14:01:03.194511 9713 raft_consensus.cc:482] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.194599 9713 raft_consensus.cc:504] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:01:03.194859 606 leader_election.cc:302] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
I20240502 14:01:03.194926 9713 leader_election.cc:288] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133:34751)
I20240502 14:01:03.195092 9582 raft_consensus.cc:2781] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 14:01:03.195147 9582 raft_consensus.cc:482] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.195188 9582 raft_consensus.cc:3037] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:03.196367 9582 raft_consensus.cc:504] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:03.196668 9582 leader_election.cc:288] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
W20240502 14:01:03.196837 477 leader_election.cc:334] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.196833 2885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72395d91ff48409693431748b3efeb6d" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4bcd7712aa844d98977a7d7e605c31f5" is_pre_election: true
I20240502 14:01:03.197396 477 leader_election.cc:302] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751; no voters: 478dfa53d7944f38b4e6cabee21b2d01, 4bcd7712aa844d98977a7d7e605c31f5
I20240502 14:01:03.197588 9713 raft_consensus.cc:2726] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 14:01:03.199003 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a192eba0af4486a212972ffdfacf9d" candidate_uuid: "58107b611f8f475a910a6fd54dfd1fb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:01:03.199100 541 raft_consensus.cc:3037] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:03.199991 541 raft_consensus.cc:2454] T 56a192eba0af4486a212972ffdfacf9d P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58107b611f8f475a910a6fd54dfd1fb9 in term 2.
I20240502 14:01:03.200289 606 leader_election.cc:302] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58107b611f8f475a910a6fd54dfd1fb9, c351a89a5fe34e788b9d8064fbffa751; no voters:
W20240502 14:01:03.202612 609 leader_election.cc:334] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
W20240502 14:01:03.202736 609 leader_election.cc:334] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.202801 9582 raft_consensus.cc:2781] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:03.202894 9582 raft_consensus.cc:686] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [term 2 LEADER]: Becoming Leader. State: Replica: 58107b611f8f475a910a6fd54dfd1fb9, State: Running, Role: LEADER
I20240502 14:01:03.203027 9582 consensus_queue.cc:237] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:03.204816 9393 raft_consensus.cc:482] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.204905 9393 raft_consensus.cc:504] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } }
I20240502 14:01:03.205267 9393 leader_election.cc:288] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:01:03.206763 32730 catalog_manager.cc:5484] T 56a192eba0af4486a212972ffdfacf9d P 58107b611f8f475a910a6fd54dfd1fb9 reported cstate change: term changed from 1 to 2, leader changed from 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) to 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "58107b611f8f475a910a6fd54dfd1fb9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "58107b611f8f475a910a6fd54dfd1fb9" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 38887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:03.207139 2954 leader_election.cc:334] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
W20240502 14:01:03.207266 2954 leader_election.cc:334] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111)
I20240502 14:01:03.207324 2954 leader_election.cc:302] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860; no voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.207468 9393 raft_consensus.cc:2726] T 1be73835d3d04112a4b34ea0a16c91d2 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 14:01:03.212766 3219 consensus_peers.cc:485] T 183e2942e969418c8e993a426935a7eb P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.214980 3219 consensus_peers.cc:485] T d9c57d60e9354df0930fcc0b3df33f86 P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.216614 409 raft_consensus.cc:1203] T 1eab750f98a6432494fa5351269e3052 P f003b732dd0f46958262b42f8d20305e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20240502 14:01:03.223129 477 consensus_peers.cc:485] T abe463eb326b4b2681af4d20e5d98698 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.231420 344 consensus_peers.cc:485] T ce81659e36874bb89da4d2268eb40aea P f003b732dd0f46958262b42f8d20305e -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.234723 609 consensus_peers.cc:485] T 59dd5ee0b3554ac493ce2d4d15a99abd P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.236282 9713 raft_consensus.cc:482] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.236373 9713 raft_consensus.cc:504] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:03.236770 9713 leader_election.cc:288] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da3f8160557e454f8547f45e80c2f013 (127.26.25.136:43131), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:03.237290 3284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9185ea12db4b413a8c76800b828836da" candidate_uuid: "c351a89a5fe34e788b9d8064fbffa751" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da3f8160557e454f8547f45e80c2f013" is_pre_election: true
W20240502 14:01:03.238624 477 leader_election.cc:334] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.238690 477 leader_election.cc:302] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751; no voters: 478dfa53d7944f38b4e6cabee21b2d01, da3f8160557e454f8547f45e80c2f013
I20240502 14:01:03.238835 9713 raft_consensus.cc:2726] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 14:01:03.242177 609 consensus_peers.cc:485] T 3f9b1351891d4690b67f8eef000a902b P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.245393 8205 raft_consensus.cc:482] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.245499 8205 raft_consensus.cc:504] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
I20240502 14:01:03.245887 8205 leader_election.cc:288] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:01:03.246937 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9185ea12db4b413a8c76800b828836da" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:01:03.247061 541 raft_consensus.cc:2454] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 1.
W20240502 14:01:03.247298 3219 leader_election.cc:334] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.247390 3216 leader_election.cc:302] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters: 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.247648 8205 raft_consensus.cc:2781] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 14:01:03.247702 8205 raft_consensus.cc:482] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.247743 8205 raft_consensus.cc:3037] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:01:03.248535 3219 consensus_peers.cc:485] T f68734d21e1843d9ad58832164dfcaae P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.248600 3219 consensus_peers.cc:485] T 8849d10396af4bf8b5ff047f93a631b1 P da3f8160557e454f8547f45e80c2f013 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.248956 8205 raft_consensus.cc:504] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
W20240502 14:01:03.249190 344 consensus_peers.cc:485] T caffa941c89c48d1b3d36ca47dc49eb2 P f003b732dd0f46958262b42f8d20305e -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.249291 8205 leader_election.cc:288] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:01:03.249445 8167 raft_consensus.cc:482] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.249529 8167 raft_consensus.cc:504] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:01:03.249877 8167 leader_election.cc:288] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
W20240502 14:01:03.250182 3219 leader_election.cc:334] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.250254 541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9185ea12db4b413a8c76800b828836da" candidate_uuid: "da3f8160557e454f8547f45e80c2f013" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:01:03.250357 541 raft_consensus.cc:3037] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:03.250355 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72395d91ff48409693431748b3efeb6d" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751" is_pre_election: true
I20240502 14:01:03.250488 540 raft_consensus.cc:2454] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 1.
W20240502 14:01:03.250855 2822 leader_election.cc:334] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.250933 2819 leader_election.cc:302] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters: 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.251084 8167 raft_consensus.cc:2781] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 14:01:03.251132 8167 raft_consensus.cc:482] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.251168 8167 raft_consensus.cc:3037] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:03.251353 541 raft_consensus.cc:2454] T 9185ea12db4b413a8c76800b828836da P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3f8160557e454f8547f45e80c2f013 in term 2.
I20240502 14:01:03.251726 3216 leader_election.cc:302] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c351a89a5fe34e788b9d8064fbffa751, da3f8160557e454f8547f45e80c2f013; no voters: 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.251950 8167 raft_consensus.cc:504] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:01:03.252290 8167 leader_election.cc:288] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 election: Requested vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251)
I20240502 14:01:03.252919 8205 raft_consensus.cc:2781] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:03.253008 8205 raft_consensus.cc:686] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [term 2 LEADER]: Becoming Leader. State: Replica: da3f8160557e454f8547f45e80c2f013, State: Running, Role: LEADER
I20240502 14:01:03.253125 8205 consensus_queue.cc:237] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } }
W20240502 14:01:03.253499 2822 leader_election.cc:334] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.253837 540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72395d91ff48409693431748b3efeb6d" candidate_uuid: "4bcd7712aa844d98977a7d7e605c31f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c351a89a5fe34e788b9d8064fbffa751"
I20240502 14:01:03.253942 540 raft_consensus.cc:3037] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:03.254951 540 raft_consensus.cc:2454] T 72395d91ff48409693431748b3efeb6d P c351a89a5fe34e788b9d8064fbffa751 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd7712aa844d98977a7d7e605c31f5 in term 2.
I20240502 14:01:03.256044 32731 catalog_manager.cc:5484] T 9185ea12db4b413a8c76800b828836da P da3f8160557e454f8547f45e80c2f013 reported cstate change: term changed from 1 to 2, leader changed from 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) to da3f8160557e454f8547f45e80c2f013 (127.26.25.136). New cstate: current_term: 2 leader_uuid: "da3f8160557e454f8547f45e80c2f013" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:03.257022 2819 leader_election.cc:302] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5, c351a89a5fe34e788b9d8064fbffa751; no voters: 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.258190 8167 raft_consensus.cc:2781] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:03.258280 8167 raft_consensus.cc:686] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 LEADER]: Becoming Leader. State: Replica: 4bcd7712aa844d98977a7d7e605c31f5, State: Running, Role: LEADER
I20240502 14:01:03.258421 8167 consensus_queue.cc:237] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } }
I20240502 14:01:03.261063 32729 catalog_manager.cc:5484] T 72395d91ff48409693431748b3efeb6d P 4bcd7712aa844d98977a7d7e605c31f5 reported cstate change: term changed from 1 to 2, leader changed from 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) to 4bcd7712aa844d98977a7d7e605c31f5 (127.26.25.133). New cstate: current_term: 2 leader_uuid: "4bcd7712aa844d98977a7d7e605c31f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c351a89a5fe34e788b9d8064fbffa751" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:03.261371 9854 raft_consensus.cc:482] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.261472 9854 raft_consensus.cc:504] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } }
I20240502 14:01:03.261839 9854 leader_election.cc:288] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
W20240502 14:01:03.264683 2954 leader_election.cc:334] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111)
W20240502 14:01:03.265821 2954 leader_election.cc:334] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.265882 2954 leader_election.cc:302] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860; no voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.266078 9854 raft_consensus.cc:2726] T 420dafd799694900b43d5df9cf4ed483 P 452ecbc085e54db49c3b01c20a96f860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 14:01:03.282181 7206 raft_consensus.cc:482] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.282297 7206 raft_consensus.cc:504] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:01:03.282653 7206 leader_election.cc:288] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
W20240502 14:01:03.284193 2822 leader_election.cc:334] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111)
W20240502 14:01:03.284305 2822 leader_election.cc:334] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.284339 2822 leader_election.cc:302] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5; no voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.284549 7206 raft_consensus.cc:2726] T fa301acea4ad4902839382b7f690f68f P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 14:01:03.290526 9590 raft_consensus.cc:482] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.290645 9590 raft_consensus.cc:504] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:01:03.291072 9590 leader_election.cc:288] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:03.291365 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860" is_pre_election: true
I20240502 14:01:03.291504 3019 raft_consensus.cc:2454] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 2.
I20240502 14:01:03.291807 341 leader_election.cc:302] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:01:03.291990 9590 raft_consensus.cc:2781] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20240502 14:01:03.292052 9590 raft_consensus.cc:482] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.292095 9590 raft_consensus.cc:3037] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:03.293179 9590 raft_consensus.cc:504] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
I20240502 14:01:03.293504 9590 leader_election.cc:288] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 3 election: Requested vote from peers 452ecbc085e54db49c3b01c20a96f860 (127.26.25.134:39061), 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443)
I20240502 14:01:03.293777 3019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094aa25b08b8498f9ae4493cafd289e3" candidate_uuid: "f003b732dd0f46958262b42f8d20305e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "452ecbc085e54db49c3b01c20a96f860"
I20240502 14:01:03.293886 3019 raft_consensus.cc:3037] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 2 FOLLOWER]: Advancing to term 3
W20240502 14:01:03.294175 344 leader_election.cc:334] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.295020 3019 raft_consensus.cc:2454] T 094aa25b08b8498f9ae4493cafd289e3 P 452ecbc085e54db49c3b01c20a96f860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f003b732dd0f46958262b42f8d20305e in term 3.
I20240502 14:01:03.295322 341 leader_election.cc:302] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452ecbc085e54db49c3b01c20a96f860, f003b732dd0f46958262b42f8d20305e; no voters:
I20240502 14:01:03.295514 9590 raft_consensus.cc:2781] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:03.295601 9590 raft_consensus.cc:686] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [term 3 LEADER]: Becoming Leader. State: Replica: f003b732dd0f46958262b42f8d20305e, State: Running, Role: LEADER
I20240502 14:01:03.295738 9590 consensus_queue.cc:237] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } }
W20240502 14:01:03.295913 344 leader_election.cc:334] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.298483 32734 catalog_manager.cc:5484] T 094aa25b08b8498f9ae4493cafd289e3 P f003b732dd0f46958262b42f8d20305e reported cstate change: term changed from 2 to 3, leader changed from 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132) to f003b732dd0f46958262b42f8d20305e (127.26.25.129). New cstate: current_term: 3 leader_uuid: "f003b732dd0f46958262b42f8d20305e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "452ecbc085e54db49c3b01c20a96f860" member_type: VOTER last_known_addr { host: "127.26.25.134" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f003b732dd0f46958262b42f8d20305e" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 44715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:03.300571 7206 raft_consensus.cc:482] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.300655 7206 raft_consensus.cc:504] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 } }
I20240502 14:01:03.301023 7206 leader_election.cc:288] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
I20240502 14:01:03.303452 8205 raft_consensus.cc:482] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 478dfa53d7944f38b4e6cabee21b2d01)
I20240502 14:01:03.303565 8205 raft_consensus.cc:504] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "478dfa53d7944f38b4e6cabee21b2d01" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 33443 } } peers { permanent_uuid: "2b6796cf0e2e496e9dd641beb432f485" member_type: VOTER last_known_addr { host: "127.26.25.135" port: 45509 } } peers { permanent_uuid: "da3f8160557e454f8547f45e80c2f013" member_type: VOTER last_known_addr { host: "127.26.25.136" port: 43131 } attrs { promote: false } }
I20240502 14:01:03.303941 8205 leader_election.cc:288] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443), 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509)
W20240502 14:01:03.304387 2822 leader_election.cc:334] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
W20240502 14:01:03.304536 2822 leader_election.cc:334] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111)
I20240502 14:01:03.304579 2822 leader_election.cc:302] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bcd7712aa844d98977a7d7e605c31f5; no voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01
W20240502 14:01:03.305219 3219 leader_election.cc:334] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111)
I20240502 14:01:03.305320 7206 raft_consensus.cc:2726] T 6787b4081a7847b8b0701b05936d441c P 4bcd7712aa844d98977a7d7e605c31f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 14:01:03.306149 3219 leader_election.cc:334] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111)
I20240502 14:01:03.306231 3219 leader_election.cc:302] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da3f8160557e454f8547f45e80c2f013; no voters: 2b6796cf0e2e496e9dd641beb432f485, 478dfa53d7944f38b4e6cabee21b2d01
I20240502 14:01:03.306488 8205 raft_consensus.cc:2726] T 167e1eb3a4034f3ca75e721b8140d757 P da3f8160557e454f8547f45e80c2f013 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 14:01:03.315850 653 tablet_service.cc:1429] Tablet server has 30 leaders and 0 scanners
I20240502 14:01:03.318353 2999 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 14:01:03.318395 672 raft_consensus.cc:1203] T 844d8895be4b4635bc21650b60049409 P 58107b611f8f475a910a6fd54dfd1fb9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:01:03.322389 2866 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 14:01:03.327911 387 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 14:01:03.332417 3263 tablet_service.cc:1429] Tablet server has 26 leaders and 0 scanners
W20240502 14:01:03.344379 2954 consensus_peers.cc:485] T 844d8895be4b4635bc21650b60049409 P 452ecbc085e54db49c3b01c20a96f860 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.345383 2885 raft_consensus.cc:1261] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P 4bcd7712aa844d98977a7d7e605c31f5 [term 2 FOLLOWER]: Refusing update from remote peer c351a89a5fe34e788b9d8064fbffa751: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 14:01:03.346055 9713 consensus_queue.cc:1035] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bcd7712aa844d98977a7d7e605c31f5" member_type: VOTER last_known_addr { host: "127.26.25.133" port: 34751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:01:03.354642 520 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
W20240502 14:01:03.375912 477 consensus_peers.cc:485] T f78ea0e379fa4f9786c7b2e45c9c8cc9 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.402011 477 consensus_peers.cc:485] T e5bed83e31be4129a19eb8c347514b80 P c351a89a5fe34e788b9d8064fbffa751 -> Peer 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): Couldn't send request to peer 478dfa53d7944f38b4e6cabee21b2d01. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.419173 2822 consensus_peers.cc:485] T d493cec70a39401b98c21446f866789a P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tools/rebalancer_tool-test.cc:1203: Failure
Value of: s.ok()
Actual: false
Expected: true
Runtime error: /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu: process exited with non-zero status 1
stdout: Per-server replica distribution summary:
Statistic | Value
-----------------------+-----------
Minimum Replica Count | 51
Maximum Replica Count | 61
Average Replica Count | 55.500000
Per-table replica distribution summary:
Replica Skew | Value
--------------+----------
Minimum | 1
Maximum | 3
Average | 2.600000
stderr: W20240502 14:00:52.396821 5382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:52.397053 5382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:52.400537 5382 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 14:00:53.102591 5382 rebalancer_tool.cc:215] running whole-cluster rebalancing
W20240502 14:00:53.457306 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table c9979d0690074e1484d971eed95aad38: could not find any suitable replica to move from server da3f8160557e454f8547f45e80c2f013 to server c351a89a5fe34e788b9d8064fbffa751
W20240502 14:00:53.457440 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 5d240759fa884da8bf576a9a2b0ef5e2: could not find any suitable replica to move from server 4bcd7712aa844d98977a7d7e605c31f5 to server c351a89a5fe34e788b9d8064fbffa751
W20240502 14:00:53.457567 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 441df53b66fd4ef7bb803ffdac4eb194: could not find any suitable replica to move from server 58107b611f8f475a910a6fd54dfd1fb9 to server da3f8160557e454f8547f45e80c2f013
W20240502 14:00:53.457669 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 43c818ed117d462dbd829b88fadb4e60: could not find any suitable replica to move from server 58107b611f8f475a910a6fd54dfd1fb9 to server da3f8160557e454f8547f45e80c2f013
W20240502 14:00:53.457911 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table ff4d6b4c29c045f2b6b5bc172c5fd4d3: could not find any suitable replica to move from server 478dfa53d7944f38b4e6cabee21b2d01 to server 4bcd7712aa844d98977a7d7e605c31f5
W20240502 14:00:53.457970 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8bbaf40cee36492e87629b1979f62041: could not find any suitable replica to move from server 2b6796cf0e2e496e9dd641beb432f485 to server da3f8160557e454f8547f45e80c2f013
W20240502 14:00:53.458000 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 5b4333df1acc4bd18638ef4d477138b9: could not find any suitable replica to move from server da3f8160557e454f8547f45e80c2f013 to server 58107b611f8f475a910a6fd54dfd1fb9
W20240502 14:00:53.458127 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table a6ece96546b04c28ab94cb79046f2008: could not find any suitable replica to move from server da3f8160557e454f8547f45e80c2f013 to server 58107b611f8f475a910a6fd54dfd1fb9
W20240502 14:00:53.458248 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 274e7365daae4e2dba024f22f983d794: could not find any suitable replica to move from server 452ecbc085e54db49c3b01c20a96f860 to server 478dfa53d7944f38b4e6cabee21b2d01
W20240502 14:00:53.458292 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table bb7573d3e8ad494cb5dd23a0f882463f: could not find any suitable replica to move from server 478dfa53d7944f38b4e6cabee21b2d01 to server f003b732dd0f46958262b42f8d20305e
W20240502 14:00:53.458341 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table f203281184c04e388bc5589f549f4a64: could not find any suitable replica to move from server 2b6796cf0e2e496e9dd641beb432f485 to server c351a89a5fe34e788b9d8064fbffa751
W20240502 14:00:53.458509 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 2b22704f8c7e447d9b7e1edd7429f8e6: could not find any suitable replica to move from server 4bcd7712aa844d98977a7d7e605c31f5 to server f003b732dd0f46958262b42f8d20305e
W20240502 14:00:53.458576 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8431fabb3f4e41f39f484115f7bf56b4: could not find any suitable replica to move from server 4bcd7712aa844d98977a7d7e605c31f5 to server 58107b611f8f475a910a6fd54dfd1fb9
W20240502 14:00:53.458614 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8dc2f81034eb43978338637236c345c6: could not find any suitable replica to move from server c351a89a5fe34e788b9d8064fbffa751 to server 2b6796cf0e2e496e9dd641beb432f485
W20240502 14:00:53.458655 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table ff4d6b4c29c045f2b6b5bc172c5fd4d3: could not find any suitable replica to move from server 58107b611f8f475a910a6fd54dfd1fb9 to server 452ecbc085e54db49c3b01c20a96f860
W20240502 14:00:53.458700 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4ce1c57cf9054191b787f9697c0c8f88: could not find any suitable replica to move from server c351a89a5fe34e788b9d8064fbffa751 to server 452ecbc085e54db49c3b01c20a96f860
W20240502 14:00:53.458776 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 147e06cde67f41129125019f57a084e5: could not find any suitable replica to move from server 58107b611f8f475a910a6fd54dfd1fb9 to server 452ecbc085e54db49c3b01c20a96f860
W20240502 14:00:53.458814 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 2b9baba891764826815587748cbcfcf1: could not find any suitable replica to move from server c351a89a5fe34e788b9d8064fbffa751 to server 2b6796cf0e2e496e9dd641beb432f485
W20240502 14:00:53.458889 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d4238514953942f2a3e7af65ce92b025: could not find any suitable replica to move from server 478dfa53d7944f38b4e6cabee21b2d01 to server 452ecbc085e54db49c3b01c20a96f860
W20240502 14:00:53.458956 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 3dec3eec8fda4142913bf2d106d579b1: could not find any suitable replica to move from server f003b732dd0f46958262b42f8d20305e to server 4bcd7712aa844d98977a7d7e605c31f5
W20240502 14:00:53.458990 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 673d46c73d60440fb2f914f93a51b362: could not find any suitable replica to move from server 58107b611f8f475a910a6fd54dfd1fb9 to server 2b6796cf0e2e496e9dd641beb432f485
W20240502 14:00:53.459038 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table c2a7fffdc4d148a5839276762398bfd9: could not find any suitable replica to move from server 452ecbc085e54db49c3b01c20a96f860 to server 58107b611f8f475a910a6fd54dfd1fb9
W20240502 14:00:53.459089 5382 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 23424c52c4e640c282457733934fe056: could not find any suitable replica to move from server 2b6796cf0e2e496e9dd641beb432f485 to server c351a89a5fe34e788b9d8064fbffa751
I20240502 14:00:53.485949 5382 rebalancer_tool.cc:1074] tablet 72395d91ff48409693431748b3efeb6d: 'da3f8160557e454f8547f45e80c2f013' -> '4bcd7712aa844d98977a7d7e605c31f5' move scheduled
I20240502 14:00:53.509346 5382 rebalancer_tool.cc:1074] tablet bbae9f2a88db4a92b89cc12dfb837ad2: '478dfa53d7944f38b4e6cabee21b2d01' -> '58107b611f8f475a910a6fd54dfd1fb9' move scheduled
I20240502 14:00:53.529829 5382 rebalancer_tool.cc:1074] tablet 4fc8d5ad25334f0a8c2efd17b5779330: '452ecbc085e54db49c3b01c20a96f860' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:00:53.548252 5382 rebalancer_tool.cc:1074] tablet 3f9b1351891d4690b67f8eef000a902b: 'c351a89a5fe34e788b9d8064fbffa751' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:53.570255 5382 rebalancer_tool.cc:1074] tablet 909bc2eb63544db0b4b3a0764be7d4a1: '4bcd7712aa844d98977a7d7e605c31f5' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:00:53.587998 5382 rebalancer_tool.cc:1074] tablet 95101f5c218746eab5095b984181a5fc: '478dfa53d7944f38b4e6cabee21b2d01' -> '452ecbc085e54db49c3b01c20a96f860' move scheduled
I20240502 14:00:53.609396 5382 rebalancer_tool.cc:1074] tablet 6423e05535aa41d5923f570c7b73f61c: '58107b611f8f475a910a6fd54dfd1fb9' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:00:53.634980 5382 rebalancer_tool.cc:1074] tablet 5f321621da5d481b8d1d2b25ddf4b395: 'c351a89a5fe34e788b9d8064fbffa751' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:53.651458 5382 rebalancer_tool.cc:1074] tablet 093babce464c4152ab6ade948dc7a616: '4bcd7712aa844d98977a7d7e605c31f5' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:00:53.671087 5382 rebalancer_tool.cc:1074] tablet b985ba4b9f534827919cbb24daa4de0e: '452ecbc085e54db49c3b01c20a96f860' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:53.695477 5382 rebalancer_tool.cc:1074] tablet 4aa562f43a154941a6e645e04757870d: 'c351a89a5fe34e788b9d8064fbffa751' -> '58107b611f8f475a910a6fd54dfd1fb9' move scheduled
I20240502 14:00:53.716500 5382 rebalancer_tool.cc:1074] tablet c47fb9f1470349b3be140e7b70811e7b: 'f003b732dd0f46958262b42f8d20305e' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:53.734241 5382 rebalancer_tool.cc:1074] tablet 6346d5ea09834bddb483da26f9bce936: '4bcd7712aa844d98977a7d7e605c31f5' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:53.752652 5382 rebalancer_tool.cc:1074] tablet 923a5d2ccd0448b381d88ced69ae8fe2: '452ecbc085e54db49c3b01c20a96f860' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:00:53.769481 5382 rebalancer_tool.cc:1074] tablet a658bde2687344beabf4d8a0ff8988e2: 'c351a89a5fe34e788b9d8064fbffa751' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:00:53.789940 5382 rebalancer_tool.cc:1074] tablet ce81659e36874bb89da4d2268eb40aea: '58107b611f8f475a910a6fd54dfd1fb9' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:53.808534 5382 rebalancer_tool.cc:1074] tablet 3bcaf05a01194e919efca9cc08d61c8f: '4bcd7712aa844d98977a7d7e605c31f5' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:53.825255 5382 rebalancer_tool.cc:1074] tablet 56a192eba0af4486a212972ffdfacf9d: '452ecbc085e54db49c3b01c20a96f860' -> 'c351a89a5fe34e788b9d8064fbffa751' move scheduled
I20240502 14:00:53.842291 5382 rebalancer_tool.cc:1074] tablet 094aa25b08b8498f9ae4493cafd289e3: 'da3f8160557e454f8547f45e80c2f013' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:00:53.869370 5382 rebalancer_tool.cc:1074] tablet fee3c05d88514648b8291e01eda2db2b: '58107b611f8f475a910a6fd54dfd1fb9' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:55.247001 5382 rebalancer_tool.cc:1136] tablet 95101f5c218746eab5095b984181a5fc: '478dfa53d7944f38b4e6cabee21b2d01' -> '452ecbc085e54db49c3b01c20a96f860' move completed: OK
I20240502 14:00:55.337256 5382 rebalancer_tool.cc:1136] tablet 4fc8d5ad25334f0a8c2efd17b5779330: '452ecbc085e54db49c3b01c20a96f860' -> 'f003b732dd0f46958262b42f8d20305e' move completed: OK
I20240502 14:00:55.347620 5382 rebalancer_tool.cc:1136] tablet 5f321621da5d481b8d1d2b25ddf4b395: 'c351a89a5fe34e788b9d8064fbffa751' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:00:55.356436 5382 rebalancer_tool.cc:1136] tablet 6346d5ea09834bddb483da26f9bce936: '4bcd7712aa844d98977a7d7e605c31f5' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:00:55.375932 5382 rebalancer_tool.cc:1136] tablet 72395d91ff48409693431748b3efeb6d: 'da3f8160557e454f8547f45e80c2f013' -> '4bcd7712aa844d98977a7d7e605c31f5' move completed: OK
I20240502 14:00:55.382160 5382 rebalancer_tool.cc:1136] tablet 4aa562f43a154941a6e645e04757870d: 'c351a89a5fe34e788b9d8064fbffa751' -> '58107b611f8f475a910a6fd54dfd1fb9' move completed: OK
I20240502 14:00:55.399966 5382 rebalancer_tool.cc:1136] tablet bbae9f2a88db4a92b89cc12dfb837ad2: '478dfa53d7944f38b4e6cabee21b2d01' -> '58107b611f8f475a910a6fd54dfd1fb9' move completed: OK
I20240502 14:00:55.412318 5382 rebalancer_tool.cc:1136] tablet 6423e05535aa41d5923f570c7b73f61c: '58107b611f8f475a910a6fd54dfd1fb9' -> 'f003b732dd0f46958262b42f8d20305e' move completed: OK
I20240502 14:00:55.430406 5382 rebalancer_tool.cc:1136] tablet 093babce464c4152ab6ade948dc7a616: '4bcd7712aa844d98977a7d7e605c31f5' -> 'da3f8160557e454f8547f45e80c2f013' move completed: OK
I20240502 14:00:55.443220 5382 rebalancer_tool.cc:1136] tablet b985ba4b9f534827919cbb24daa4de0e: '452ecbc085e54db49c3b01c20a96f860' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:00:55.451864 5382 rebalancer_tool.cc:1136] tablet c47fb9f1470349b3be140e7b70811e7b: 'f003b732dd0f46958262b42f8d20305e' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:00:55.471531 5382 rebalancer_tool.cc:1136] tablet ce81659e36874bb89da4d2268eb40aea: '58107b611f8f475a910a6fd54dfd1fb9' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:00:55.499222 5382 rebalancer_tool.cc:1074] tablet be795fbe277b4e9fac8561d6589b7c05: '478dfa53d7944f38b4e6cabee21b2d01' -> '4bcd7712aa844d98977a7d7e605c31f5' move scheduled
I20240502 14:00:55.518391 5382 rebalancer_tool.cc:1074] tablet 1bdc7b131596496aa6779885fe5d9433: '58107b611f8f475a910a6fd54dfd1fb9' -> '452ecbc085e54db49c3b01c20a96f860' move scheduled
I20240502 14:00:55.535149 5382 rebalancer_tool.cc:1074] tablet 7113a6da4229422099dbd604a560fb95: 'f003b732dd0f46958262b42f8d20305e' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:55.553011 5382 rebalancer_tool.cc:1074] tablet 792bc86bad314a2bbf4bba3001c78894: '478dfa53d7944f38b4e6cabee21b2d01' -> 'c351a89a5fe34e788b9d8064fbffa751' move scheduled
I20240502 14:00:55.574368 5382 rebalancer_tool.cc:1074] tablet e5bed83e31be4129a19eb8c347514b80: '58107b611f8f475a910a6fd54dfd1fb9' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:00:55.604617 5382 rebalancer_tool.cc:1074] tablet ad4e1b6dea1e4beeb4b82852ded83c5f: '4bcd7712aa844d98977a7d7e605c31f5' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:00:55.644124 5382 rebalancer_tool.cc:1074] tablet caffa941c89c48d1b3d36ca47dc49eb2: '452ecbc085e54db49c3b01c20a96f860' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:55.679874 5382 rebalancer_tool.cc:1074] tablet a176c4678de94ccc93b14d04fc46ea90: '478dfa53d7944f38b4e6cabee21b2d01' -> 'c351a89a5fe34e788b9d8064fbffa751' move scheduled
I20240502 14:00:55.704021 5382 rebalancer_tool.cc:1074] tablet 9486274cd39b4de18c37ab9f71f261c2: '58107b611f8f475a910a6fd54dfd1fb9' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:00:55.730794 5382 rebalancer_tool.cc:1074] tablet e93a3a3fca9042e9b8bdf3c60ec73a20: '4bcd7712aa844d98977a7d7e605c31f5' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:00:55.759327 5382 rebalancer_tool.cc:1074] tablet 10d6cc99575f4ebb8c980e9cb3e35145: '452ecbc085e54db49c3b01c20a96f860' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:55.814139 5382 rebalancer_tool.cc:1136] tablet fee3c05d88514648b8291e01eda2db2b: '58107b611f8f475a910a6fd54dfd1fb9' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:00:55.820060 5382 rebalancer_tool.cc:1136] tablet 56a192eba0af4486a212972ffdfacf9d: '452ecbc085e54db49c3b01c20a96f860' -> 'c351a89a5fe34e788b9d8064fbffa751' move completed: OK
I20240502 14:00:55.832506 5382 rebalancer_tool.cc:1136] tablet 3bcaf05a01194e919efca9cc08d61c8f: '4bcd7712aa844d98977a7d7e605c31f5' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:00:55.838889 5382 rebalancer_tool.cc:1136] tablet 3f9b1351891d4690b67f8eef000a902b: 'c351a89a5fe34e788b9d8064fbffa751' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:00:55.909207 5382 rebalancer_tool.cc:1136] tablet a658bde2687344beabf4d8a0ff8988e2: 'c351a89a5fe34e788b9d8064fbffa751' -> 'f003b732dd0f46958262b42f8d20305e' move completed: OK
I20240502 14:00:55.925318 5382 rebalancer_tool.cc:1074] tablet 58a0cf9cd92a43ca8c481964707eda6d: 'c351a89a5fe34e788b9d8064fbffa751' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:55.949266 5382 rebalancer_tool.cc:1074] tablet 8568aa57fce64ccf89a36deb5b7b0f3b: '58107b611f8f475a910a6fd54dfd1fb9' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:55.969290 5382 rebalancer_tool.cc:1074] tablet 184c03808a1e470dba9c6bfa8e04774a: '452ecbc085e54db49c3b01c20a96f860' -> 'c351a89a5fe34e788b9d8064fbffa751' move scheduled
I20240502 14:00:55.988001 5382 rebalancer_tool.cc:1074] tablet 564ba4de769e45af8de10fe8c86eef31: 'f003b732dd0f46958262b42f8d20305e' -> '4bcd7712aa844d98977a7d7e605c31f5' move scheduled
I20240502 14:00:56.010509 5382 rebalancer_tool.cc:1074] tablet 59dd5ee0b3554ac493ce2d4d15a99abd: '478dfa53d7944f38b4e6cabee21b2d01' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:56.646407 5382 rebalancer_tool.cc:1136] tablet 923a5d2ccd0448b381d88ced69ae8fe2: '452ecbc085e54db49c3b01c20a96f860' -> 'da3f8160557e454f8547f45e80c2f013' move completed: OK
I20240502 14:00:56.690819 5382 rebalancer_tool.cc:1074] tablet 85ad896b8838472096851682071c0a9c: '58107b611f8f475a910a6fd54dfd1fb9' -> '452ecbc085e54db49c3b01c20a96f860' move scheduled
I20240502 14:00:56.717000 5382 rebalancer_tool.cc:1074] tablet 7e9f86a6149d43f3a217769865d9a3d9: 'c351a89a5fe34e788b9d8064fbffa751' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:00:56.856438 5382 rebalancer_tool.cc:1136] tablet 909bc2eb63544db0b4b3a0764be7d4a1: '4bcd7712aa844d98977a7d7e605c31f5' -> 'da3f8160557e454f8547f45e80c2f013' move completed: OK
I20240502 14:00:56.867866 5382 rebalancer_tool.cc:1136] tablet 094aa25b08b8498f9ae4493cafd289e3: 'da3f8160557e454f8547f45e80c2f013' -> 'f003b732dd0f46958262b42f8d20305e' move completed: OK
I20240502 14:00:56.895377 5382 rebalancer_tool.cc:1074] tablet edcd3af1a04c43d3ab79d539b758ecb0: 'f003b732dd0f46958262b42f8d20305e' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:00:56.984027 5382 rebalancer_tool.cc:1136] tablet 792bc86bad314a2bbf4bba3001c78894: '478dfa53d7944f38b4e6cabee21b2d01' -> 'c351a89a5fe34e788b9d8064fbffa751' move completed: OK
I20240502 14:00:57.098937 5382 rebalancer_tool.cc:1074] tablet db0bf9f6821045e1a259703fc3445226: '4bcd7712aa844d98977a7d7e605c31f5' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:57.206976 5382 rebalancer_tool.cc:1074] tablet 618c1b9af9504c03bd250d88c3a2b0ba: 'da3f8160557e454f8547f45e80c2f013' -> 'c351a89a5fe34e788b9d8064fbffa751' move scheduled
I20240502 14:00:57.376570 5382 rebalancer_tool.cc:1136] tablet 564ba4de769e45af8de10fe8c86eef31: 'f003b732dd0f46958262b42f8d20305e' -> '4bcd7712aa844d98977a7d7e605c31f5' move completed: OK
I20240502 14:00:57.402256 5382 rebalancer_tool.cc:1136] tablet 58a0cf9cd92a43ca8c481964707eda6d: 'c351a89a5fe34e788b9d8064fbffa751' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:00:57.413516 5382 rebalancer_tool.cc:1136] tablet 9486274cd39b4de18c37ab9f71f261c2: '58107b611f8f475a910a6fd54dfd1fb9' -> 'da3f8160557e454f8547f45e80c2f013' move completed: OK
I20240502 14:00:57.429952 5382 rebalancer_tool.cc:1136] tablet 10d6cc99575f4ebb8c980e9cb3e35145: '452ecbc085e54db49c3b01c20a96f860' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:00:57.436224 5382 rebalancer_tool.cc:1136] tablet caffa941c89c48d1b3d36ca47dc49eb2: '452ecbc085e54db49c3b01c20a96f860' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:00:57.443641 5382 rebalancer_tool.cc:1136] tablet 1bdc7b131596496aa6779885fe5d9433: '58107b611f8f475a910a6fd54dfd1fb9' -> '452ecbc085e54db49c3b01c20a96f860' move completed: OK
I20240502 14:00:57.467787 5382 rebalancer_tool.cc:1136] tablet be795fbe277b4e9fac8561d6589b7c05: '478dfa53d7944f38b4e6cabee21b2d01' -> '4bcd7712aa844d98977a7d7e605c31f5' move completed: OK
I20240502 14:00:57.486003 5382 rebalancer_tool.cc:1136] tablet a176c4678de94ccc93b14d04fc46ea90: '478dfa53d7944f38b4e6cabee21b2d01' -> 'c351a89a5fe34e788b9d8064fbffa751' move completed: OK
I20240502 14:00:57.509383 5382 rebalancer_tool.cc:1136] tablet ad4e1b6dea1e4beeb4b82852ded83c5f: '4bcd7712aa844d98977a7d7e605c31f5' -> 'f003b732dd0f46958262b42f8d20305e' move completed: OK
I20240502 14:00:57.524480 5382 rebalancer_tool.cc:1136] tablet e93a3a3fca9042e9b8bdf3c60ec73a20: '4bcd7712aa844d98977a7d7e605c31f5' -> 'f003b732dd0f46958262b42f8d20305e' move completed: OK
I20240502 14:00:57.547571 5382 rebalancer_tool.cc:1074] tablet 2df438b3678241d9a6aa6f0deb2cee2d: '4bcd7712aa844d98977a7d7e605c31f5' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:57.582041 5382 rebalancer_tool.cc:1074] tablet 2b4469ea792f4c1f9dce98e47b51f9f4: '452ecbc085e54db49c3b01c20a96f860' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:00:57.632021 5382 rebalancer_tool.cc:1074] tablet 9cd00d821b5d4b35ada028a8c83d0ed7: '4bcd7712aa844d98977a7d7e605c31f5' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:57.661507 5382 rebalancer_tool.cc:1074] tablet ee25d04c50a64ab98282e3659c94bd71: '58107b611f8f475a910a6fd54dfd1fb9' -> '452ecbc085e54db49c3b01c20a96f860' move scheduled
I20240502 14:00:57.700284 5382 rebalancer_tool.cc:1074] tablet c9d974d536ed41bb9cdc0c3abb64a773: 'c351a89a5fe34e788b9d8064fbffa751' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:57.723356 5382 rebalancer_tool.cc:1074] tablet 2ffed38329ca4e339a849c5ab3b09e8b: '2b6796cf0e2e496e9dd641beb432f485' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:00:57.749034 5382 rebalancer_tool.cc:1074] tablet a4015371eba240cb8e8f7e2180ac3d27: '4bcd7712aa844d98977a7d7e605c31f5' -> '58107b611f8f475a910a6fd54dfd1fb9' move scheduled
I20240502 14:00:57.777748 5382 rebalancer_tool.cc:1074] tablet e673d6e79c0f4c62a393a542473930b1: 'c351a89a5fe34e788b9d8064fbffa751' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:00:57.797677 5382 rebalancer_tool.cc:1074] tablet c1dc8816aa514d1abcce63ce0e0bf357: '452ecbc085e54db49c3b01c20a96f860' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:57.897671 5382 rebalancer_tool.cc:1136] tablet 59dd5ee0b3554ac493ce2d4d15a99abd: '478dfa53d7944f38b4e6cabee21b2d01' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:00:57.904258 5382 rebalancer_tool.cc:1136] tablet 184c03808a1e470dba9c6bfa8e04774a: '452ecbc085e54db49c3b01c20a96f860' -> 'c351a89a5fe34e788b9d8064fbffa751' move completed: OK
I20240502 14:00:57.916332 5382 rebalancer_tool.cc:1136] tablet 8568aa57fce64ccf89a36deb5b7b0f3b: '58107b611f8f475a910a6fd54dfd1fb9' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:00:57.951783 5382 rebalancer_tool.cc:1136] tablet 7113a6da4229422099dbd604a560fb95: 'f003b732dd0f46958262b42f8d20305e' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:00:57.968358 5382 rebalancer_tool.cc:1074] tablet 8cf2d71983204c3c96d112cab1004478: '2b6796cf0e2e496e9dd641beb432f485' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:00:57.987874 5382 rebalancer_tool.cc:1074] tablet 9a9aac11ac0b44339b6d9b0f84702871: '2b6796cf0e2e496e9dd641beb432f485' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:58.015482 5382 rebalancer_tool.cc:1074] tablet 49595e1db3c44669b438bd0ac1093a62: '4bcd7712aa844d98977a7d7e605c31f5' -> 'c351a89a5fe34e788b9d8064fbffa751' move scheduled
I20240502 14:00:58.100899 5382 rebalancer_tool.cc:1074] tablet 2d9d248c2c6745489d2656752db9ddd5: '452ecbc085e54db49c3b01c20a96f860' -> '58107b611f8f475a910a6fd54dfd1fb9' move scheduled
I20240502 14:00:58.133965 5382 rebalancer_tool.cc:1074] tablet 2f2c438f3a854f528ae8555cdb13721b: '2b6796cf0e2e496e9dd641beb432f485' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:00:58.284139 5382 rebalancer_tool.cc:1136] tablet 7e9f86a6149d43f3a217769865d9a3d9: 'c351a89a5fe34e788b9d8064fbffa751' -> 'da3f8160557e454f8547f45e80c2f013' move completed: OK
I20240502 14:00:58.298403 5382 rebalancer_tool.cc:1136] tablet 85ad896b8838472096851682071c0a9c: '58107b611f8f475a910a6fd54dfd1fb9' -> '452ecbc085e54db49c3b01c20a96f860' move completed: OK
I20240502 14:00:58.319300 5382 rebalancer_tool.cc:1074] tablet 567df9d664b7423785175fa385f83e53: 'c351a89a5fe34e788b9d8064fbffa751' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:00:58.337123 5382 rebalancer_tool.cc:1074] tablet f51e005446c64a07b18f0dbf5056afbc: '58107b611f8f475a910a6fd54dfd1fb9' -> '452ecbc085e54db49c3b01c20a96f860' move scheduled
I20240502 14:00:58.486879 5382 rebalancer_tool.cc:1136] tablet edcd3af1a04c43d3ab79d539b758ecb0: 'f003b732dd0f46958262b42f8d20305e' -> 'da3f8160557e454f8547f45e80c2f013' move completed: OK
I20240502 14:00:58.532531 5382 rebalancer_tool.cc:1136] tablet e5bed83e31be4129a19eb8c347514b80: '58107b611f8f475a910a6fd54dfd1fb9' -> 'da3f8160557e454f8547f45e80c2f013' move completed: OK
I20240502 14:00:58.555907 5382 rebalancer_tool.cc:1074] tablet b20d05c7433e4f56832233e2fc9adc84: 'da3f8160557e454f8547f45e80c2f013' -> '58107b611f8f475a910a6fd54dfd1fb9' move scheduled
I20240502 14:00:58.759522 5382 rebalancer_tool.cc:1136] tablet db0bf9f6821045e1a259703fc3445226: '4bcd7712aa844d98977a7d7e605c31f5' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:00:58.777734 5382 rebalancer_tool.cc:1136] tablet 618c1b9af9504c03bd250d88c3a2b0ba: 'da3f8160557e454f8547f45e80c2f013' -> 'c351a89a5fe34e788b9d8064fbffa751' move completed: OK
I20240502 14:00:58.801575 5382 rebalancer_tool.cc:1074] tablet 5e922f7cee8046e7b4263dd57b8d1185: 'da3f8160557e454f8547f45e80c2f013' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:58.848001 5382 rebalancer_tool.cc:1074] tablet 638cb0f08414491ea80139eb51406dfe: 'f003b732dd0f46958262b42f8d20305e' -> 'c351a89a5fe34e788b9d8064fbffa751' move scheduled
I20240502 14:00:59.305554 5382 rebalancer_tool.cc:1136] tablet ee25d04c50a64ab98282e3659c94bd71: '58107b611f8f475a910a6fd54dfd1fb9' -> '452ecbc085e54db49c3b01c20a96f860' move completed: OK
I20240502 14:00:59.333499 5382 rebalancer_tool.cc:1136] tablet 9cd00d821b5d4b35ada028a8c83d0ed7: '4bcd7712aa844d98977a7d7e605c31f5' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:00:59.349480 5382 rebalancer_tool.cc:1136] tablet 2df438b3678241d9a6aa6f0deb2cee2d: '4bcd7712aa844d98977a7d7e605c31f5' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:00:59.384617 5382 rebalancer_tool.cc:1136] tablet c9d974d536ed41bb9cdc0c3abb64a773: 'c351a89a5fe34e788b9d8064fbffa751' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:00:59.399094 5382 rebalancer_tool.cc:1136] tablet 2b4469ea792f4c1f9dce98e47b51f9f4: '452ecbc085e54db49c3b01c20a96f860' -> 'f003b732dd0f46958262b42f8d20305e' move completed: OK
I20240502 14:00:59.420831 5382 rebalancer_tool.cc:1074] tablet 72590739196d48ac88d69b5bcd1124b0: '4bcd7712aa844d98977a7d7e605c31f5' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:59.445366 5382 rebalancer_tool.cc:1074] tablet 9e642731d7684e33b82eabf97c380123: '452ecbc085e54db49c3b01c20a96f860' -> '58107b611f8f475a910a6fd54dfd1fb9' move scheduled
I20240502 14:00:59.467227 5382 rebalancer_tool.cc:1074] tablet 987eabba58cd472696e64f30db468c71: 'c351a89a5fe34e788b9d8064fbffa751' -> '4bcd7712aa844d98977a7d7e605c31f5' move scheduled
I20240502 14:00:59.532737 5382 rebalancer_tool.cc:1074] tablet 00d3e2b37ce743bf90e211f5385efe90: 'da3f8160557e454f8547f45e80c2f013' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:00:59.559784 5382 rebalancer_tool.cc:1074] tablet 81cd2e05295641678fa8d7711c797113: 'f003b732dd0f46958262b42f8d20305e' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:00:59.681452 5382 rebalancer_tool.cc:1136] tablet 49595e1db3c44669b438bd0ac1093a62: '4bcd7712aa844d98977a7d7e605c31f5' -> 'c351a89a5fe34e788b9d8064fbffa751' move completed: OK
I20240502 14:00:59.728567 5382 rebalancer_tool.cc:1136] tablet 9a9aac11ac0b44339b6d9b0f84702871: '2b6796cf0e2e496e9dd641beb432f485' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:00:59.762225 5382 rebalancer_tool.cc:1136] tablet e673d6e79c0f4c62a393a542473930b1: 'c351a89a5fe34e788b9d8064fbffa751' -> 'da3f8160557e454f8547f45e80c2f013' move completed: OK
I20240502 14:00:59.777601 5382 rebalancer_tool.cc:1136] tablet 2ffed38329ca4e339a849c5ab3b09e8b: '2b6796cf0e2e496e9dd641beb432f485' -> 'f003b732dd0f46958262b42f8d20305e' move completed: OK
I20240502 14:00:59.783351 5382 rebalancer_tool.cc:1136] tablet 2d9d248c2c6745489d2656752db9ddd5: '452ecbc085e54db49c3b01c20a96f860' -> '58107b611f8f475a910a6fd54dfd1fb9' move completed: OK
I20240502 14:00:59.789250 5382 rebalancer_tool.cc:1136] tablet c1dc8816aa514d1abcce63ce0e0bf357: '452ecbc085e54db49c3b01c20a96f860' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:00:59.806638 5382 rebalancer_tool.cc:1074] tablet ec443abbb4954bf3b28336e0b5bc0c4b: '452ecbc085e54db49c3b01c20a96f860' -> 'c351a89a5fe34e788b9d8064fbffa751' move scheduled
I20240502 14:00:59.825165 5382 rebalancer_tool.cc:1074] tablet be6290bc7df6465f8a43aae431b34f8b: '4bcd7712aa844d98977a7d7e605c31f5' -> '58107b611f8f475a910a6fd54dfd1fb9' move scheduled
I20240502 14:00:59.845392 5382 rebalancer_tool.cc:1074] tablet 79db2b3bbcb041f6be0d86b150a3d324: '2b6796cf0e2e496e9dd641beb432f485' -> '452ecbc085e54db49c3b01c20a96f860' move scheduled
I20240502 14:00:59.878110 5382 rebalancer_tool.cc:1074] tablet 08f30aab835c4380b8b7a01400e5aa55: '478dfa53d7944f38b4e6cabee21b2d01' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:00:59.901342 5382 rebalancer_tool.cc:1074] tablet 9185ea12db4b413a8c76800b828836da: 'f003b732dd0f46958262b42f8d20305e' -> 'c351a89a5fe34e788b9d8064fbffa751' move scheduled
I20240502 14:00:59.935896 5382 rebalancer_tool.cc:1074] tablet 4c27c9fb983c49a59a2f84776775e1bd: '2b6796cf0e2e496e9dd641beb432f485' -> '452ecbc085e54db49c3b01c20a96f860' move scheduled
I20240502 14:01:00.083087 5382 rebalancer_tool.cc:1136] tablet f51e005446c64a07b18f0dbf5056afbc: '58107b611f8f475a910a6fd54dfd1fb9' -> '452ecbc085e54db49c3b01c20a96f860' move completed: OK
I20240502 14:01:00.092203 5382 rebalancer_tool.cc:1136] tablet 567df9d664b7423785175fa385f83e53: 'c351a89a5fe34e788b9d8064fbffa751' -> 'da3f8160557e454f8547f45e80c2f013' move completed: OK
I20240502 14:01:00.111091 5382 rebalancer_tool.cc:1136] tablet 2f2c438f3a854f528ae8555cdb13721b: '2b6796cf0e2e496e9dd641beb432f485' -> 'f003b732dd0f46958262b42f8d20305e' move completed: OK
I20240502 14:01:00.133565 5382 rebalancer_tool.cc:1136] tablet 8cf2d71983204c3c96d112cab1004478: '2b6796cf0e2e496e9dd641beb432f485' -> 'f003b732dd0f46958262b42f8d20305e' move completed: OK
I20240502 14:01:00.140954 5382 rebalancer_tool.cc:1136] tablet a4015371eba240cb8e8f7e2180ac3d27: '4bcd7712aa844d98977a7d7e605c31f5' -> '58107b611f8f475a910a6fd54dfd1fb9' move completed: OK
I20240502 14:01:00.163901 5382 rebalancer_tool.cc:1074] tablet c2ce30696a474d23818c3f70cc86e9dd: '2b6796cf0e2e496e9dd641beb432f485' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:01:00.192680 5382 rebalancer_tool.cc:1074] tablet 10c70369a07e45afb7bd885b58daec40: '4bcd7712aa844d98977a7d7e605c31f5' -> '58107b611f8f475a910a6fd54dfd1fb9' move scheduled
I20240502 14:01:00.215116 5382 rebalancer_tool.cc:1074] tablet f3d9ae2ed9a84c2c928dc1d03f325fa3: '452ecbc085e54db49c3b01c20a96f860' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:01:00.233515 5382 rebalancer_tool.cc:1074] tablet 167e1eb3a4034f3ca75e721b8140d757: 'c351a89a5fe34e788b9d8064fbffa751' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:01:00.260232 5382 rebalancer_tool.cc:1074] tablet 604579a2517d4c2eaf855db299d15d5e: '2b6796cf0e2e496e9dd641beb432f485' -> '58107b611f8f475a910a6fd54dfd1fb9' move scheduled
I20240502 14:01:00.337229 5382 rebalancer_tool.cc:1136] tablet 638cb0f08414491ea80139eb51406dfe: 'f003b732dd0f46958262b42f8d20305e' -> 'c351a89a5fe34e788b9d8064fbffa751' move completed: OK
I20240502 14:01:00.474108 5382 rebalancer_tool.cc:1074] tablet 7450ad74adcc49c3820f93b568ee7c21: 'c351a89a5fe34e788b9d8064fbffa751' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:01:00.574326 5382 rebalancer_tool.cc:1136] tablet 5e922f7cee8046e7b4263dd57b8d1185: 'da3f8160557e454f8547f45e80c2f013' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:01:00.672362 5382 rebalancer_tool.cc:1074] tablet adaf3fcf84de4223876c7601bdfd45ea: 'f003b732dd0f46958262b42f8d20305e' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:01:00.932792 5382 rebalancer_tool.cc:1136] tablet 9e642731d7684e33b82eabf97c380123: '452ecbc085e54db49c3b01c20a96f860' -> '58107b611f8f475a910a6fd54dfd1fb9' move completed: OK
I20240502 14:01:01.015309 5382 rebalancer_tool.cc:1136] tablet 00d3e2b37ce743bf90e211f5385efe90: 'da3f8160557e454f8547f45e80c2f013' -> '2b6796cf0e2e496e9dd641beb432f485' move completed: OK
I20240502 14:01:01.108938 5382 rebalancer_tool.cc:1136] tablet 72590739196d48ac88d69b5bcd1124b0: '4bcd7712aa844d98977a7d7e605c31f5' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:01:01.125895 5382 rebalancer_tool.cc:1136] tablet 81cd2e05295641678fa8d7711c797113: 'f003b732dd0f46958262b42f8d20305e' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:01:01.132416 5382 rebalancer_tool.cc:1136] tablet 987eabba58cd472696e64f30db468c71: 'c351a89a5fe34e788b9d8064fbffa751' -> '4bcd7712aa844d98977a7d7e605c31f5' move completed: OK
I20240502 14:01:01.148464 5382 rebalancer_tool.cc:1074] tablet 556b3a4cab9245d4939ba4c10bf9d4b4: '58107b611f8f475a910a6fd54dfd1fb9' -> '4bcd7712aa844d98977a7d7e605c31f5' move scheduled
I20240502 14:01:01.165764 5382 rebalancer_tool.cc:1074] tablet c80435eecda64ecea7864522f5a1dd4c: 'da3f8160557e454f8547f45e80c2f013' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:01:01.187099 5382 rebalancer_tool.cc:1074] tablet f47f72fcaaae46d68f4cdab7595095ca: 'c351a89a5fe34e788b9d8064fbffa751' -> '4bcd7712aa844d98977a7d7e605c31f5' move scheduled
I20240502 14:01:01.211736 5382 rebalancer_tool.cc:1074] tablet d493cec70a39401b98c21446f866789a: '452ecbc085e54db49c3b01c20a96f860' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:01:01.231734 5382 rebalancer_tool.cc:1074] tablet 20937116074f456ea732396f80bd7d6a: 'f003b732dd0f46958262b42f8d20305e' -> '478dfa53d7944f38b4e6cabee21b2d01' move scheduled
I20240502 14:01:01.348328 5382 rebalancer_tool.cc:1136] tablet 4c27c9fb983c49a59a2f84776775e1bd: '2b6796cf0e2e496e9dd641beb432f485' -> '452ecbc085e54db49c3b01c20a96f860' move completed: OK
I20240502 14:01:01.354208 5382 rebalancer_tool.cc:1136] tablet 9185ea12db4b413a8c76800b828836da: 'f003b732dd0f46958262b42f8d20305e' -> 'c351a89a5fe34e788b9d8064fbffa751' move completed: OK
I20240502 14:01:01.377473 5382 rebalancer_tool.cc:1136] tablet 79db2b3bbcb041f6be0d86b150a3d324: '2b6796cf0e2e496e9dd641beb432f485' -> '452ecbc085e54db49c3b01c20a96f860' move completed: OK
I20240502 14:01:01.417294 5382 rebalancer_tool.cc:1136] tablet ec443abbb4954bf3b28336e0b5bc0c4b: '452ecbc085e54db49c3b01c20a96f860' -> 'c351a89a5fe34e788b9d8064fbffa751' move completed: OK
I20240502 14:01:01.451398 5382 rebalancer_tool.cc:1074] tablet 527784cdca2f4840aa2f2a373a2c6db6: '2b6796cf0e2e496e9dd641beb432f485' -> '452ecbc085e54db49c3b01c20a96f860' move scheduled
I20240502 14:01:01.471534 5382 rebalancer_tool.cc:1074] tablet 0dc1880244a64b9f96ad5b8122a49fce: '452ecbc085e54db49c3b01c20a96f860' -> 'c351a89a5fe34e788b9d8064fbffa751' move scheduled
I20240502 14:01:01.495505 5382 rebalancer_tool.cc:1074] tablet c8e843cd7c984ebfb532fec67f97e249: 'da3f8160557e454f8547f45e80c2f013' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:01:01.555172 5382 rebalancer_tool.cc:1074] tablet 8689975c1d9d41f4ae7152bf6e4a58a8: 'c351a89a5fe34e788b9d8064fbffa751' -> '4bcd7712aa844d98977a7d7e605c31f5' move scheduled
I20240502 14:01:01.581758 5382 rebalancer_tool.cc:1074] tablet 9bf2e05fba1943edb5cd00989f93df7c: 'f003b732dd0f46958262b42f8d20305e' -> '452ecbc085e54db49c3b01c20a96f860' move scheduled
I20240502 14:01:01.825995 5382 rebalancer_tool.cc:1136] tablet 604579a2517d4c2eaf855db299d15d5e: '2b6796cf0e2e496e9dd641beb432f485' -> '58107b611f8f475a910a6fd54dfd1fb9' move completed: OK
W20240502 14:01:02.094956 9446 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 14:01:01.829122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.25.132:33443 (local address 127.0.0.1:51460)
0502 14:01:01.829409 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0502 14:01:01.829422 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0502 14:01:01.829512 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 14:01:02.094097 (+264585us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":94,"threads_started":1}
I20240502 14:01:02.097180 5382 rebalancer_tool.cc:1123] tablet 167e1eb3a4034f3ca75e721b8140d757: 'c351a89a5fe34e788b9d8064fbffa751' -> 'da3f8160557e454f8547f45e80c2f013' move is abandoned: Network error: Client connection negotiation failed: client connection to 127.26.25.132:33443: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 14:01:02.118064 5382 rebalancer_tool.cc:1136] tablet 08f30aab835c4380b8b7a01400e5aa55: '478dfa53d7944f38b4e6cabee21b2d01' -> 'da3f8160557e454f8547f45e80c2f013' move completed: OK
I20240502 14:01:02.130016 5382 rebalancer_tool.cc:1136] tablet f3d9ae2ed9a84c2c928dc1d03f325fa3: '452ecbc085e54db49c3b01c20a96f860' -> '478dfa53d7944f38b4e6cabee21b2d01' move completed: OK
I20240502 14:01:02.141377 5382 rebalancer_tool.cc:1136] tablet be6290bc7df6465f8a43aae431b34f8b: '4bcd7712aa844d98977a7d7e605c31f5' -> '58107b611f8f475a910a6fd54dfd1fb9' move completed: OK
I20240502 14:01:02.147701 5382 rebalancer_tool.cc:1136] tablet c2ce30696a474d23818c3f70cc86e9dd: '2b6796cf0e2e496e9dd641beb432f485' -> 'f003b732dd0f46958262b42f8d20305e' move completed: OK
I20240502 14:01:02.154851 5382 rebalancer_tool.cc:1136] tablet b20d05c7433e4f56832233e2fc9adc84: 'da3f8160557e454f8547f45e80c2f013' -> '58107b611f8f475a910a6fd54dfd1fb9' move completed: OK
I20240502 14:01:02.160883 5382 rebalancer_tool.cc:1136] tablet 10c70369a07e45afb7bd885b58daec40: '4bcd7712aa844d98977a7d7e605c31f5' -> '58107b611f8f475a910a6fd54dfd1fb9' move completed: OK
I20240502 14:01:02.174345 5382 rebalancer_tool.cc:1074] tablet 1415dfd1daed4e1a8a00439c77185a55: '58107b611f8f475a910a6fd54dfd1fb9' -> '2b6796cf0e2e496e9dd641beb432f485' move scheduled
I20240502 14:01:02.210711 5382 rebalancer_tool.cc:1074] tablet 544ed46acc5345f58031444b9a2ef776: 'da3f8160557e454f8547f45e80c2f013' -> '58107b611f8f475a910a6fd54dfd1fb9' move scheduled
I20240502 14:01:02.239084 5382 rebalancer_tool.cc:1074] tablet a35103d97b874010b36c1863b2258c33: '478dfa53d7944f38b4e6cabee21b2d01' -> 'da3f8160557e454f8547f45e80c2f013' move scheduled
I20240502 14:01:02.267206 5382 rebalancer_tool.cc:1074] tablet 02a36ffbdd0f4d68953aa441c04b42eb: '58107b611f8f475a910a6fd54dfd1fb9' -> '4bcd7712aa844d98977a7d7e605c31f5' move scheduled
I20240502 14:01:02.307416 5382 rebalancer_tool.cc:1074] tablet 1eab750f98a6432494fa5351269e3052: '452ecbc085e54db49c3b01c20a96f860' -> 'f003b732dd0f46958262b42f8d20305e' move scheduled
I20240502 14:01:02.327937 5382 rebalancer_tool.cc:1074] tablet 844d8895be4b4635bc21650b60049409: '2b6796cf0e2e496e9dd641beb432f485' -> '58107b611f8f475a910a6fd54dfd1fb9' move scheduled
I20240502 14:01:03.036775 5382 rebalancer_tool.cc:1123] tablet adaf3fcf84de4223876c7601bdfd45ea: 'f003b732dd0f46958262b42f8d20305e' -> '478dfa53d7944f38b4e6cabee21b2d01' move is abandoned: Network error: recv got EOF from 127.26.25.135:45509 (error 108)
I20240502 14:01:03.039624 5382 rebalancer_tool.cc:1123] tablet 7450ad74adcc49c3820f93b568ee7c21: 'c351a89a5fe34e788b9d8064fbffa751' -> 'f003b732dd0f46958262b42f8d20305e' move is abandoned: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111)
I20240502 14:01:03.104435 5382 rebalancer_tool.cc:1123] tablet 9bf2e05fba1943edb5cd00989f93df7c: 'f003b732dd0f46958262b42f8d20305e' -> '452ecbc085e54db49c3b01c20a96f860' move is abandoned: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111)
I20240502 14:01:03.112097 5382 rebalancer_tool.cc:756] re-synchronizing cluster state
Illegal state: tablet server 478dfa53d7944f38b4e6cabee21b2d01 (127.26.25.132:33443): unacceptable health status UNAVAILABLE
I20240502 14:01:03.422996 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 326
W20240502 14:01:03.432426 2822 consensus_peers.cc:485] T 10d6cc99575f4ebb8c980e9cb3e35145 P 4bcd7712aa844d98977a7d7e605c31f5 -> Peer 2b6796cf0e2e496e9dd641beb432f485 (127.26.25.135:45509): Couldn't send request to peer 2b6796cf0e2e496e9dd641beb432f485. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.135:45509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.434900 608 connection.cc:535] client connection to 127.26.25.129:44715 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 14:01:03.435025 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 459
W20240502 14:01:03.437764 3218 consensus_peers.cc:485] T 2f2c438f3a854f528ae8555cdb13721b P da3f8160557e454f8547f45e80c2f013 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.129:44715: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.440250 608 consensus_peers.cc:485] T a658bde2687344beabf4d8a0ff8988e2 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.129:44715: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.440855 608 consensus_peers.cc:485] T 08f30aab835c4380b8b7a01400e5aa55 P 58107b611f8f475a910a6fd54dfd1fb9 -> Peer f003b732dd0f46958262b42f8d20305e (127.26.25.129:44715): Couldn't send request to peer f003b732dd0f46958262b42f8d20305e. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.129:44715: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.445024 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 591
I20240502 14:01:03.454545 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 2803
W20240502 14:01:03.459585 2951 consensus_peers.cc:485] T 1eab750f98a6432494fa5351269e3052 P 452ecbc085e54db49c3b01c20a96f860 -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.130:35251: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.460366 2951 consensus_peers.cc:485] T a37ad1cd10564bcebe788fb8c49a66a8 P 452ecbc085e54db49c3b01c20a96f860 -> Peer 58107b611f8f475a910a6fd54dfd1fb9 (127.26.25.131:38887): Couldn't send request to peer 58107b611f8f475a910a6fd54dfd1fb9. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.131:38887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:03.461714 2951 consensus_peers.cc:485] T 75c2963776af487b9ff7ee3004fcaa07 P 452ecbc085e54db49c3b01c20a96f860 -> Peer c351a89a5fe34e788b9d8064fbffa751 (127.26.25.130:35251): Couldn't send request to peer c351a89a5fe34e788b9d8064fbffa751. Status: Network error: Client connection negotiation failed: client connection to 127.26.25.130:35251: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:03.463660 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 2934
I20240502 14:01:03.471366 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 3201
I20240502 14:01:03.479125 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 32703
2024-05-02T14:01:03Z chronyd exiting
I20240502 14:01:03.496671 26726 test_util.cc:175] -----------------------------------------------
I20240502 14:01:03.496716 26726 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_1.1714658379150170-26726-0
[ FAILED ] DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing/1, where GetParam() = 4-byte object <01-00 00-00> (59433 ms)
[----------] 1 test from DDLDuringRebalancingTest (59433 ms total)
[----------] 1 test from RebalancingDuringElectionStormTest
[ RUN ] RebalancingDuringElectionStormTest.RoundRobin/1
I20240502 14:01:03.497380 26726 test_util.cc:268] Using random seed: 144751778
I20240502 14:01:03.498106 26726 ts_itest-base.cc:112] Starting cluster with:
I20240502 14:01:03.498142 26726 ts_itest-base.cc:113] --------------
I20240502 14:01:03.498162 26726 ts_itest-base.cc:114] 8 tablet servers
I20240502 14:01:03.498178 26726 ts_itest-base.cc:115] 3 replicas per TS
I20240502 14:01:03.498203 26726 ts_itest-base.cc:116] --------------
2024-05-02T14:01:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T14:01:03Z Disabled control of system clock
I20240502 14:01:03.505981 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.25.190:39653
--webserver_interface=127.26.25.190
--webserver_port=0
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.25.190:39653
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240502 14:01:03.605695 9941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:03.605898 9941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:03.605937 9941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:03.608871 9941 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:03.608930 9941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 14:01:03.608959 9941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:03.608999 9941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 14:01:03.609032 9941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 14:01:03.612568 9941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.26.25.190:39653
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.25.190:39653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.26.25.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.9941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:03.613343 9941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:03.614375 9941 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:03.621752 9946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:03.622269 9947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:03.623131 9949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:03.623982 9941 server_base.cc:1034] running on GCE node
I20240502 14:01:03.624425 9941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:03.625221 9941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:03.626385 9941 hybrid_clock.cc:648] HybridClock initialized: now 1714658463626356 us; error 42 us; skew 500 ppm
I20240502 14:01:03.628212 9941 webserver.cc:460] Webserver started at http://127.26.25.190:35563/ using document root <none> and password file <none>
I20240502 14:01:03.628712 9941 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:03.628772 9941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:03.628944 9941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:03.630467 9941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "3aa77e8e1a4845b89168449e89d79506"
format_stamp: "Formatted at 2024-05-02 14:01:03 on dist-test-slave-sldb"
I20240502 14:01:03.630899 9941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "3aa77e8e1a4845b89168449e89d79506"
format_stamp: "Formatted at 2024-05-02 14:01:03 on dist-test-slave-sldb"
I20240502 14:01:03.633985 9941 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240502 14:01:03.636061 9955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:03.636969 9941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 14:01:03.637084 9941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
uuid: "3aa77e8e1a4845b89168449e89d79506"
format_stamp: "Formatted at 2024-05-02 14:01:03 on dist-test-slave-sldb"
I20240502 14:01:03.637161 9941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:03.653563 9941 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:03.654083 9941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:03.654227 9941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:03.660159 9941 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.190:39653
I20240502 14:01:03.660172 10007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.190:39653 every 8 connection(s)
I20240502 14:01:03.661015 9941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 14:01:03.663336 10008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:03.667995 10008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506: Bootstrap starting.
I20240502 14:01:03.669994 10008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:03.670606 10008 log.cc:824] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:03.671181 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 9941
I20240502 14:01:03.671330 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 14:01:03.672317 10008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506: No bootstrap required, opened a new log
I20240502 14:01:03.675020 10008 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa77e8e1a4845b89168449e89d79506" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 39653 } }
I20240502 14:01:03.675181 10008 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:03.675212 10008 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa77e8e1a4845b89168449e89d79506, State: Initialized, Role: FOLLOWER
I20240502 14:01:03.675592 10008 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa77e8e1a4845b89168449e89d79506" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 39653 } }
I20240502 14:01:03.675719 10008 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:01:03.675786 10008 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:01:03.675918 10008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:03.676574 10008 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa77e8e1a4845b89168449e89d79506" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 39653 } }
I20240502 14:01:03.676877 10008 leader_election.cc:302] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3aa77e8e1a4845b89168449e89d79506; no voters:
I20240502 14:01:03.677126 10008 leader_election.cc:288] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:01:03.677210 10013 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:03.678141 10013 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [term 1 LEADER]: Becoming Leader. State: Replica: 3aa77e8e1a4845b89168449e89d79506, State: Running, Role: LEADER
I20240502 14:01:03.678282 10008 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 14:01:03.678418 10013 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa77e8e1a4845b89168449e89d79506" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 39653 } }
I20240502 14:01:03.679812 10015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3aa77e8e1a4845b89168449e89d79506. Latest consensus state: current_term: 1 leader_uuid: "3aa77e8e1a4845b89168449e89d79506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa77e8e1a4845b89168449e89d79506" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 39653 } } }
I20240502 14:01:03.679965 10015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [sys.catalog]: This master's current role is: LEADER
I20240502 14:01:03.680248 10014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3aa77e8e1a4845b89168449e89d79506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3aa77e8e1a4845b89168449e89d79506" member_type: VOTER last_known_addr { host: "127.26.25.190" port: 39653 } } }
I20240502 14:01:03.680317 10014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506 [sys.catalog]: This master's current role is: LEADER
W20240502 14:01:03.682983 10028 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 14:01:03.683046 10028 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 14:01:03.683125 10026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 14:01:03.683926 10026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 14:01:03.688472 10026 catalog_manager.cc:1347] Generated new cluster ID: 645d823d64484aff9c529d1ff35ce7df
I20240502 14:01:03.688535 10026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 14:01:03.696311 10026 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 14:01:03.697371 10026 catalog_manager.cc:1504] Loading token signing keys...
I20240502 14:01:03.704908 10026 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 3aa77e8e1a4845b89168449e89d79506: Generated new TSK 0
I20240502 14:01:03.705559 10026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 14:01:03.712378 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.129:0
--local_ip_for_outbound_sockets=127.26.25.129
--webserver_interface=127.26.25.129
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:01:03.817253 10032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:03.817432 10032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:03.817472 10032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:03.820883 10032 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:03.820945 10032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:03.821022 10032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.129
I20240502 14:01:03.824527 10032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.26.25.129
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.10032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.129
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:03.825294 10032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:03.826306 10032 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:03.834946 10037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:03.835477 10038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:03.836081 10040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:03.836984 10032 server_base.cc:1034] running on GCE node
I20240502 14:01:03.837338 10032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:03.837831 10032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:03.838987 10032 hybrid_clock.cc:648] HybridClock initialized: now 1714658463838964 us; error 42 us; skew 500 ppm
I20240502 14:01:03.840765 10032 webserver.cc:460] Webserver started at http://127.26.25.129:39677/ using document root <none> and password file <none>
I20240502 14:01:03.841282 10032 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:03.841337 10032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:03.841480 10032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:03.842973 10032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "068f58913ea3478599df829d5118f7be"
format_stamp: "Formatted at 2024-05-02 14:01:03 on dist-test-slave-sldb"
I20240502 14:01:03.843430 10032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "068f58913ea3478599df829d5118f7be"
format_stamp: "Formatted at 2024-05-02 14:01:03 on dist-test-slave-sldb"
I20240502 14:01:03.846622 10032 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 14:01:03.848667 10046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:03.849612 10032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:03.849730 10032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "068f58913ea3478599df829d5118f7be"
format_stamp: "Formatted at 2024-05-02 14:01:03 on dist-test-slave-sldb"
I20240502 14:01:03.849817 10032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:03.886119 10032 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:03.886771 10032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:03.886951 10032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:03.887492 10032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:03.888361 10032 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:03.888410 10032 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:03.888469 10032 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:03.888501 10032 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:03.896437 10032 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.129:38571
I20240502 14:01:03.896477 10159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.129:38571 every 8 connection(s)
I20240502 14:01:03.897356 10032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 14:01:03.897960 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 10032
I20240502 14:01:03.898072 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 14:01:03.900355 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.130:0
--local_ip_for_outbound_sockets=127.26.25.130
--webserver_interface=127.26.25.130
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:03.906997 10160 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:03.907192 10160 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:03.907660 10160 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:03.909013 9972 ts_manager.cc:194] Registered new tserver with Master: 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:03.910102 9972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.129:54885
W20240502 14:01:04.002964 10163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:04.003155 10163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:04.003198 10163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:04.006093 10163 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:04.006148 10163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:04.006239 10163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.130
I20240502 14:01:04.009598 10163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.26.25.130
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.10163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.130
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:04.010334 10163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:04.011354 10163 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:04.019388 10169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:04.019989 10171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:04.020649 10163 server_base.cc:1034] running on GCE node
W20240502 14:01:04.021360 10168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:04.021764 10163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:04.022261 10163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:04.023445 10163 hybrid_clock.cc:648] HybridClock initialized: now 1714658464023430 us; error 27 us; skew 500 ppm
I20240502 14:01:04.025218 10163 webserver.cc:460] Webserver started at http://127.26.25.130:43251/ using document root <none> and password file <none>
I20240502 14:01:04.025718 10163 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:04.025803 10163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:04.025985 10163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:04.027472 10163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "1ca5103abf454aaca375e9e07eb139cc"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.027951 10163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "1ca5103abf454aaca375e9e07eb139cc"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.030941 10163 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:04.032954 10177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.033854 10163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:04.033955 10163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "1ca5103abf454aaca375e9e07eb139cc"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.034030 10163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:04.048718 10163 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:04.049216 10163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:04.049363 10163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:04.049826 10163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:04.050608 10163 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:04.050657 10163 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.050710 10163 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:04.050743 10163 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.059067 10163 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.130:39739
I20240502 14:01:04.059111 10290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.130:39739 every 8 connection(s)
I20240502 14:01:04.059926 10163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 14:01:04.066478 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 10163
I20240502 14:01:04.066597 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 14:01:04.067867 10291 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:04.068071 10291 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:04.068562 10291 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:04.068778 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.131:0
--local_ip_for_outbound_sockets=127.26.25.131
--webserver_interface=127.26.25.131
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:04.069525 9971 ts_manager.cc:194] Registered new tserver with Master: 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:04.070261 9971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.130:39127
W20240502 14:01:04.171984 10294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:04.172185 10294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:04.172257 10294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:04.175189 10294 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:04.175252 10294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:04.175346 10294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.131
I20240502 14:01:04.178809 10294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.26.25.131
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.10294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.131
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:04.179702 10294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:04.180840 10294 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:04.188889 10299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:04.189460 10302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:04.190335 10300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:04.191113 10294 server_base.cc:1034] running on GCE node
I20240502 14:01:04.191448 10294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:04.191959 10294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:04.193112 10294 hybrid_clock.cc:648] HybridClock initialized: now 1714658464193104 us; error 27 us; skew 500 ppm
I20240502 14:01:04.194813 10294 webserver.cc:460] Webserver started at http://127.26.25.131:33501/ using document root <none> and password file <none>
I20240502 14:01:04.195300 10294 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:04.195359 10294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:04.195518 10294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:04.197110 10294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "243b53aabd2d4443a8971e707bcc4a51"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.197546 10294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "243b53aabd2d4443a8971e707bcc4a51"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.200589 10294 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:01:04.202498 10308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.203348 10294 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:04.203447 10294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "243b53aabd2d4443a8971e707bcc4a51"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.203523 10294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:04.218716 10294 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:04.219182 10294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:04.219328 10294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:04.219755 10294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:04.220547 10294 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:04.220595 10294 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.220652 10294 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:04.220680 10294 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.228335 10294 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.131:40365
I20240502 14:01:04.228372 10421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.131:40365 every 8 connection(s)
I20240502 14:01:04.229117 10294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 14:01:04.234081 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 10294
I20240502 14:01:04.234185 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 14:01:04.236476 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.132:0
--local_ip_for_outbound_sockets=127.26.25.132
--webserver_interface=127.26.25.132
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:04.237185 10422 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:04.237381 10422 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:04.237841 10422 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:04.238826 9971 ts_manager.cc:194] Registered new tserver with Master: 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:04.239569 9971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.131:40953
W20240502 14:01:04.343446 10425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:04.343652 10425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:04.343693 10425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:04.346879 10425 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:04.346953 10425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:04.347019 10425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.132
I20240502 14:01:04.350647 10425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.26.25.132
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.10425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.132
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:04.351438 10425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:04.352514 10425 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:04.360649 10431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:04.361459 10430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:04.362200 10433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:04.363057 10425 server_base.cc:1034] running on GCE node
I20240502 14:01:04.363421 10425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:04.363942 10425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:04.365084 10425 hybrid_clock.cc:648] HybridClock initialized: now 1714658464365069 us; error 29 us; skew 500 ppm
I20240502 14:01:04.366940 10425 webserver.cc:460] Webserver started at http://127.26.25.132:32799/ using document root <none> and password file <none>
I20240502 14:01:04.367470 10425 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:04.367525 10425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:04.367694 10425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:04.369330 10425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.369825 10425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.373142 10425 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20240502 14:01:04.375198 10439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.376236 10425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:04.376349 10425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.376461 10425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:04.401827 10425 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:04.402376 10425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:04.402530 10425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:04.402997 10425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:04.403810 10425 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:04.403877 10425 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.403936 10425 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:04.403967 10425 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.412439 10425 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.132:38293
I20240502 14:01:04.412492 10553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.132:38293 every 8 connection(s)
I20240502 14:01:04.413286 10425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 14:01:04.421480 10554 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:04.421672 10554 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:04.422142 10554 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:04.422662 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 10425
I20240502 14:01:04.422768 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 14:01:04.423130 9971 ts_manager.cc:194] Registered new tserver with Master: ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:04.423744 9971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.132:41487
I20240502 14:01:04.424775 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.133:0
--local_ip_for_outbound_sockets=127.26.25.133
--webserver_interface=127.26.25.133
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:01:04.526491 10557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:04.526665 10557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:04.526706 10557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:04.529506 10557 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:04.529561 10557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:04.529639 10557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.133
I20240502 14:01:04.532994 10557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.25.133
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.10557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.133
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:04.533702 10557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:04.534677 10557 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:04.543268 10562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:04.544008 10563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:04.544682 10565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:04.545328 10557 server_base.cc:1034] running on GCE node
I20240502 14:01:04.545709 10557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:04.546187 10557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:04.547341 10557 hybrid_clock.cc:648] HybridClock initialized: now 1714658464547337 us; error 26 us; skew 500 ppm
I20240502 14:01:04.549101 10557 webserver.cc:460] Webserver started at http://127.26.25.133:35167/ using document root <none> and password file <none>
I20240502 14:01:04.549626 10557 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:04.549685 10557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:04.549844 10557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:04.551319 10557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "5a19a740e4ae4be8955282aced7210f5"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.551752 10557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "5a19a740e4ae4be8955282aced7210f5"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.554952 10557 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240502 14:01:04.556911 10571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.557828 10557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:04.557916 10557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "5a19a740e4ae4be8955282aced7210f5"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.558008 10557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:04.569696 10557 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:04.570166 10557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:04.570302 10557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:04.570724 10557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:04.571499 10557 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:04.571545 10557 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.571586 10557 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:04.571617 10557 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.579351 10557 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.133:42411
I20240502 14:01:04.579464 10684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.133:42411 every 8 connection(s)
I20240502 14:01:04.580260 10557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 14:01:04.588212 10685 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:04.588418 10685 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:04.588874 10685 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:04.589807 9972 ts_manager.cc:194] Registered new tserver with Master: 5a19a740e4ae4be8955282aced7210f5 (127.26.25.133:42411)
I20240502 14:01:04.590096 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 10557
I20240502 14:01:04.590209 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 14:01:04.590464 9972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.133:38419
I20240502 14:01:04.592339 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.134:0
--local_ip_for_outbound_sockets=127.26.25.134
--webserver_interface=127.26.25.134
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:01:04.694674 10688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:04.694917 10688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:04.694983 10688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:04.698063 10688 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:04.698129 10688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:04.698217 10688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.134
I20240502 14:01:04.701814 10688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.26.25.134
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.10688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.134
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:04.702735 10688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:04.703869 10688 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:04.712340 10693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:04.712934 10696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:04.713722 10694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:04.714505 10688 server_base.cc:1034] running on GCE node
I20240502 14:01:04.715008 10688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:04.715543 10688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:04.716687 10688 hybrid_clock.cc:648] HybridClock initialized: now 1714658464716675 us; error 26 us; skew 500 ppm
I20240502 14:01:04.718437 10688 webserver.cc:460] Webserver started at http://127.26.25.134:34949/ using document root <none> and password file <none>
I20240502 14:01:04.718937 10688 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:04.718986 10688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:04.719144 10688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:04.720755 10688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "f1bfaf3d95f54984a2b1f97a23352440"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.721208 10688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "f1bfaf3d95f54984a2b1f97a23352440"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.724452 10688 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:01:04.726512 10702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.727481 10688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:04.727587 10688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "f1bfaf3d95f54984a2b1f97a23352440"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.727672 10688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:04.745116 10688 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:04.745661 10688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:04.745810 10688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:04.746295 10688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:04.747169 10688 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:04.747216 10688 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.747272 10688 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:04.747301 10688 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.755376 10688 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.134:34253
I20240502 14:01:04.755396 10815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.134:34253 every 8 connection(s)
I20240502 14:01:04.756309 10688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 14:01:04.757395 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 10688
I20240502 14:01:04.757524 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 14:01:04.759616 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.135:0
--local_ip_for_outbound_sockets=127.26.25.135
--webserver_interface=127.26.25.135
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:04.765724 10816 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:04.765995 10816 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:04.766677 10816 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:04.767844 9972 ts_manager.cc:194] Registered new tserver with Master: f1bfaf3d95f54984a2b1f97a23352440 (127.26.25.134:34253)
I20240502 14:01:04.768658 9972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.134:47567
W20240502 14:01:04.868534 10819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:04.868750 10819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:04.868793 10819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:04.871753 10819 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:04.871834 10819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:04.871901 10819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.135
I20240502 14:01:04.875514 10819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.26.25.135
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.10819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.135
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:04.876416 10819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:04.877664 10819 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:04.886560 10827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:04.887372 10825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:04.887991 10824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:04.888753 10819 server_base.cc:1034] running on GCE node
I20240502 14:01:04.889097 10819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:04.889629 10819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:04.890787 10819 hybrid_clock.cc:648] HybridClock initialized: now 1714658464890777 us; error 23 us; skew 500 ppm
I20240502 14:01:04.892660 10819 webserver.cc:460] Webserver started at http://127.26.25.135:33039/ using document root <none> and password file <none>
I20240502 14:01:04.893193 10819 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:04.893249 10819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:04.893441 10819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:04.895098 10819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "e3d80b4f40cf438d9d0a63f54a1f5907"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.895555 10819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "e3d80b4f40cf438d9d0a63f54a1f5907"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.898965 10819 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 14:01:04.901253 10833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.902263 10819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:04.902364 10819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "e3d80b4f40cf438d9d0a63f54a1f5907"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:04.902462 10819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:04.912055 10160 heartbeater.cc:503] Master 127.26.25.190:39653 was elected leader, sending a full tablet report...
I20240502 14:01:04.916396 10819 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:04.916968 10819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:04.917119 10819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:04.917631 10819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:04.918485 10819 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:04.918535 10819 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.918581 10819 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:04.918632 10819 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:04.927114 10819 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.135:38811
I20240502 14:01:04.927139 10946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.135:38811 every 8 connection(s)
I20240502 14:01:04.928050 10819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 14:01:04.935801 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 10819
I20240502 14:01:04.935959 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 14:01:04.936734 10947 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:04.936985 10947 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:04.937603 10947 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:04.938361 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.136:0
--local_ip_for_outbound_sockets=127.26.25.136
--webserver_interface=127.26.25.136
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:04.938694 9972 ts_manager.cc:194] Registered new tserver with Master: e3d80b4f40cf438d9d0a63f54a1f5907 (127.26.25.135:38811)
I20240502 14:01:04.939242 9972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.135:36389
W20240502 14:01:05.047727 10950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:05.048105 10950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:05.048180 10950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:05.051440 10950 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:05.051527 10950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:05.051640 10950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.136
I20240502 14:01:05.055581 10950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.26.25.136
--webserver_port=0
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.10950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.136
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:05.056563 10950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:05.057698 10950 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:05.067195 10955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:05.067991 10956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:05.068765 10958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:05.069432 10950 server_base.cc:1034] running on GCE node
I20240502 14:01:05.069834 10950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:05.070425 10950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:05.071589 10950 hybrid_clock.cc:648] HybridClock initialized: now 1714658465071575 us; error 35 us; skew 500 ppm
I20240502 14:01:05.072067 10291 heartbeater.cc:503] Master 127.26.25.190:39653 was elected leader, sending a full tablet report...
I20240502 14:01:05.074188 10950 webserver.cc:460] Webserver started at http://127.26.25.136:43337/ using document root <none> and password file <none>
I20240502 14:01:05.074927 10950 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:05.075016 10950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:05.075263 10950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:05.077024 10950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "1ca6438d01864c25bf5eb0a54955e8a9"
format_stamp: "Formatted at 2024-05-02 14:01:05 on dist-test-slave-sldb"
I20240502 14:01:05.077458 10950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "1ca6438d01864c25bf5eb0a54955e8a9"
format_stamp: "Formatted at 2024-05-02 14:01:05 on dist-test-slave-sldb"
I20240502 14:01:05.081013 10950 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:01:05.083216 10964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:05.084291 10950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:01:05.084390 10950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "1ca6438d01864c25bf5eb0a54955e8a9"
format_stamp: "Formatted at 2024-05-02 14:01:05 on dist-test-slave-sldb"
I20240502 14:01:05.084472 10950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:05.102646 10950 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:05.103260 10950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:05.103399 10950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:05.103948 10950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:05.104844 10950 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:05.104892 10950 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:05.104940 10950 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:05.104961 10950 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:05.113523 10950 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.136:34019
I20240502 14:01:05.113552 11077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.136:34019 every 8 connection(s)
I20240502 14:01:05.114436 10950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 14:01:05.115417 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 10950
I20240502 14:01:05.115528 26726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 14:01:05.122784 11078 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:05.123013 11078 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:05.123534 11078 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:05.124511 9972 ts_manager.cc:194] Registered new tserver with Master: 1ca6438d01864c25bf5eb0a54955e8a9 (127.26.25.136:34019)
I20240502 14:01:05.125056 9972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.136:35841
I20240502 14:01:05.129582 26726 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 14:01:05.138484 26726 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 14:01:05.138553 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 10557
I20240502 14:01:05.144172 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 10688
I20240502 14:01:05.149962 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 10819
I20240502 14:01:05.155162 26726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu with pid 10950
I20240502 14:01:05.241719 10422 heartbeater.cc:503] Master 127.26.25.190:39653 was elected leader, sending a full tablet report...
I20240502 14:01:05.425736 10554 heartbeater.cc:503] Master 127.26.25.190:39653 was elected leader, sending a full tablet report...
I20240502 14:01:08.912137 9972 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44294:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:01:08.933207 10487 tablet_service.cc:1467] Processing CreateTablet for tablet 9a76fe53d2dd4b12bc04ac52089a7120 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3d5668e4a84e4279ba0ec4339dcfba76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:08.933208 10224 tablet_service.cc:1467] Processing CreateTablet for tablet c211d47d28ae45ed9bdf0471b553edb1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3d5668e4a84e4279ba0ec4339dcfba76]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:08.933207 10092 tablet_service.cc:1467] Processing CreateTablet for tablet c211d47d28ae45ed9bdf0471b553edb1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3d5668e4a84e4279ba0ec4339dcfba76]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:08.933434 10093 tablet_service.cc:1467] Processing CreateTablet for tablet bb0d4740d5ca4d80a80751151b3f57a7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3d5668e4a84e4279ba0ec4339dcfba76]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:08.934201 10224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c211d47d28ae45ed9bdf0471b553edb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:08.934201 10093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb0d4740d5ca4d80a80751151b3f57a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:08.934202 10487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a76fe53d2dd4b12bc04ac52089a7120. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:08.934299 10225 tablet_service.cc:1467] Processing CreateTablet for tablet bb0d4740d5ca4d80a80751151b3f57a7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3d5668e4a84e4279ba0ec4339dcfba76]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:08.934527 10225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb0d4740d5ca4d80a80751151b3f57a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:08.935788 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c211d47d28ae45ed9bdf0471b553edb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:08.936079 10486 tablet_service.cc:1467] Processing CreateTablet for tablet c211d47d28ae45ed9bdf0471b553edb1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3d5668e4a84e4279ba0ec4339dcfba76]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:08.936321 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c211d47d28ae45ed9bdf0471b553edb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:08.933208 10094 tablet_service.cc:1467] Processing CreateTablet for tablet 9a76fe53d2dd4b12bc04ac52089a7120 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3d5668e4a84e4279ba0ec4339dcfba76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:08.937377 10094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a76fe53d2dd4b12bc04ac52089a7120. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:08.941561 11099 tablet_bootstrap.cc:492] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:08.943262 11100 tablet_bootstrap.cc:492] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:08.943923 11099 tablet_bootstrap.cc:654] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:08.944165 10356 tablet_service.cc:1467] Processing CreateTablet for tablet 9a76fe53d2dd4b12bc04ac52089a7120 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3d5668e4a84e4279ba0ec4339dcfba76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:08.944164 10355 tablet_service.cc:1467] Processing CreateTablet for tablet bb0d4740d5ca4d80a80751151b3f57a7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=3d5668e4a84e4279ba0ec4339dcfba76]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:08.944787 11099 log.cc:824] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:08.945242 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb0d4740d5ca4d80a80751151b3f57a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:08.945596 10356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a76fe53d2dd4b12bc04ac52089a7120. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:08.945904 11100 tablet_bootstrap.cc:654] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:08.946810 11098 tablet_bootstrap.cc:492] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:08.946947 11100 log.cc:824] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:08.948506 11099 tablet_bootstrap.cc:492] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:08.948698 11099 ts_tablet_manager.cc:1367] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240502 14:01:08.948830 11098 tablet_bootstrap.cc:654] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:08.949604 11098 log.cc:824] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:08.951234 11098 tablet_bootstrap.cc:492] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:08.951418 11098 ts_tablet_manager.cc:1367] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 14:01:08.951905 11104 tablet_bootstrap.cc:492] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:08.952785 11099 raft_consensus.cc:348] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:08.953004 11099 raft_consensus.cc:374] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:08.953069 11099 raft_consensus.cc:729] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:08.953495 11099 consensus_queue.cc:260] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:08.954169 11104 tablet_bootstrap.cc:654] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:08.954371 11099 ts_tablet_manager.cc:1398] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240502 14:01:08.954592 11099 tablet_bootstrap.cc:492] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:08.954978 11104 log.cc:824] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:08.955541 11099 tablet_bootstrap.cc:654] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:08.955425 11098 raft_consensus.cc:348] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.955603 11098 raft_consensus.cc:374] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:08.955651 11098 raft_consensus.cc:729] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:08.956073 11098 consensus_queue.cc:260] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.956789 11098 ts_tablet_manager.cc:1398] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240502 14:01:08.957973 11100 tablet_bootstrap.cc:492] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:08.958173 11100 ts_tablet_manager.cc:1367] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20240502 14:01:08.958211 11098 tablet_bootstrap.cc:492] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:08.958317 11104 tablet_bootstrap.cc:492] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:08.958518 11104 ts_tablet_manager.cc:1367] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20240502 14:01:08.959172 11098 tablet_bootstrap.cc:654] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:08.960489 11098 tablet_bootstrap.cc:492] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:08.960597 11098 ts_tablet_manager.cc:1367] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:08.961150 11098 raft_consensus.cc:348] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:08.961282 11098 raft_consensus.cc:374] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:08.961321 11098 raft_consensus.cc:729] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:08.961508 11098 consensus_queue.cc:260] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:08.961788 11098 ts_tablet_manager.cc:1398] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:08.962070 11100 raft_consensus.cc:348] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:08.962301 11100 raft_consensus.cc:374] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:08.962361 11100 raft_consensus.cc:729] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:08.962627 11099 tablet_bootstrap.cc:492] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:08.962721 11099 ts_tablet_manager.cc:1367] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240502 14:01:08.962802 11100 consensus_queue.cc:260] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:08.963165 11104 raft_consensus.cc:348] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.963284 11099 raft_consensus.cc:348] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.963407 11099 raft_consensus.cc:374] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:08.963449 11099 raft_consensus.cc:729] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:08.963467 11104 raft_consensus.cc:374] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:08.963541 11104 raft_consensus.cc:729] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:08.963579 11099 consensus_queue.cc:260] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.963717 11100 ts_tablet_manager.cc:1398] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240502 14:01:08.963887 11099 ts_tablet_manager.cc:1398] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:08.963994 11104 consensus_queue.cc:260] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.964118 11100 tablet_bootstrap.cc:492] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:08.964825 11104 ts_tablet_manager.cc:1398] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 14:01:08.965178 11104 tablet_bootstrap.cc:492] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:08.966277 11100 tablet_bootstrap.cc:654] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:08.967449 11104 tablet_bootstrap.cc:654] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:08.967546 11100 tablet_bootstrap.cc:492] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:08.967645 11100 ts_tablet_manager.cc:1367] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:08.968264 11100 raft_consensus.cc:348] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.968407 11100 raft_consensus.cc:374] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:08.968451 11100 raft_consensus.cc:729] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:08.968623 11104 tablet_bootstrap.cc:492] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:08.968605 11100 consensus_queue.cc:260] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.968714 11104 ts_tablet_manager.cc:1367] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 14:01:08.968880 11100 ts_tablet_manager.cc:1398] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:08.969048 11100 tablet_bootstrap.cc:492] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:08.969198 11104 raft_consensus.cc:348] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.969291 11104 raft_consensus.cc:374] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:08.969336 11104 raft_consensus.cc:729] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:08.969460 11104 consensus_queue.cc:260] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.969708 11104 ts_tablet_manager.cc:1398] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:08.969844 11100 tablet_bootstrap.cc:654] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:08.970963 11100 tablet_bootstrap.cc:492] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:08.971056 11100 ts_tablet_manager.cc:1367] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:08.971459 11100 raft_consensus.cc:348] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.971570 11100 raft_consensus.cc:374] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:08.971611 11100 raft_consensus.cc:729] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:08.971732 11100 consensus_queue.cc:260] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:08.972034 11100 ts_tablet_manager.cc:1398] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 14:01:08.983044 10423 tablet.cc:2369] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:01:08.999006 11108 raft_consensus.cc:482] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:08.999233 11108 raft_consensus.cc:504] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:09.000483 11108 leader_election.cc:288] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:09.005784 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c211d47d28ae45ed9bdf0471b553edb1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:09.006100 10114 raft_consensus.cc:2454] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:09.006068 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c211d47d28ae45ed9bdf0471b553edb1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:09.006330 10508 raft_consensus.cc:2454] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:09.006608 10180 leader_election.cc:302] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:09.006888 11108 raft_consensus.cc:2781] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:09.006959 11108 raft_consensus.cc:482] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:09.006990 11108 raft_consensus.cc:3037] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.008019 11108 raft_consensus.cc:504] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:09.008363 11108 leader_election.cc:288] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:09.008847 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c211d47d28ae45ed9bdf0471b553edb1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:09.008850 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c211d47d28ae45ed9bdf0471b553edb1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:09.008983 10508 raft_consensus.cc:3037] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.008993 10114 raft_consensus.cc:3037] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.010231 10508 raft_consensus.cc:2454] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:09.010535 10114 raft_consensus.cc:2454] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:09.010546 10180 leader_election.cc:302] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:09.010713 11108 raft_consensus.cc:2781] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:09.010944 11108 raft_consensus.cc:686] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:09.011260 11108 consensus_queue.cc:237] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:09.014322 9970 catalog_manager.cc:5484] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 1 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:09.064330 10292 tablet.cc:2369] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 14:01:09.152182 10161 tablet.cc:2369] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 14:01:09.167429 10555 tablet.cc:2369] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:01:09.208408 11111 raft_consensus.cc:482] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:09.208628 11111 raft_consensus.cc:504] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.209769 11111 leader_election.cc:288] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:09.213526 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a76fe53d2dd4b12bc04ac52089a7120" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:09.213591 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a76fe53d2dd4b12bc04ac52089a7120" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:09.213658 10114 raft_consensus.cc:2454] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:09.213747 10508 raft_consensus.cc:2454] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:09.214044 10311 leader_election.cc:302] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:09.214313 11111 raft_consensus.cc:2781] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:09.214381 11111 raft_consensus.cc:482] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:09.214413 11111 raft_consensus.cc:3037] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.215293 11111 raft_consensus.cc:504] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.215605 11111 leader_election.cc:288] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:09.216012 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a76fe53d2dd4b12bc04ac52089a7120" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:09.216058 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a76fe53d2dd4b12bc04ac52089a7120" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:09.216112 10114 raft_consensus.cc:3037] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.216172 10508 raft_consensus.cc:3037] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.216832 10114 raft_consensus.cc:2454] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:09.216946 10508 raft_consensus.cc:2454] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:09.217125 10311 leader_election.cc:302] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:09.217360 11111 raft_consensus.cc:2781] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:09.217550 11111 raft_consensus.cc:686] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:09.217844 11111 consensus_queue.cc:237] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.220677 9970 catalog_manager.cc:5484] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:09.330929 11108 raft_consensus.cc:482] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:09.331055 11108 raft_consensus.cc:504] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.331519 11108 leader_election.cc:288] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:09.332100 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d4740d5ca4d80a80751151b3f57a7" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:09.332250 10114 raft_consensus.cc:2454] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:09.332576 10180 leader_election.cc:302] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:09.332738 11108 raft_consensus.cc:2781] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:09.332804 11108 raft_consensus.cc:482] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:09.332850 11108 raft_consensus.cc:3037] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.333840 11108 raft_consensus.cc:504] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.334156 11108 leader_election.cc:288] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:09.334543 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d4740d5ca4d80a80751151b3f57a7" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:09.334659 10114 raft_consensus.cc:3037] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.335391 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d4740d5ca4d80a80751151b3f57a7" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:09.335404 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d4740d5ca4d80a80751151b3f57a7" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:09.335600 10376 raft_consensus.cc:2454] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:09.335688 10114 raft_consensus.cc:2454] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:09.335978 10180 leader_election.cc:302] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:09.336128 11108 raft_consensus.cc:2781] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:09.336205 11108 raft_consensus.cc:686] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:09.336347 11108 consensus_queue.cc:237] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.338270 9970 catalog_manager.cc:5484] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 1 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:09.476547 11108 consensus_queue.cc:1035] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:09.482504 11117 consensus_queue.cc:1035] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 14:01:09.489279 11124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:09.489559 11124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:09.492689 11124 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 14:01:09.578835 10375 raft_consensus.cc:3037] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.584026 10375 raft_consensus.cc:1261] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:09.584233 10114 raft_consensus.cc:1261] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:09.584689 11117 consensus_queue.cc:1035] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:09.584880 11108 consensus_queue.cc:1035] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:09.588954 10508 raft_consensus.cc:1261] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:09.589287 10114 raft_consensus.cc:1261] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:09.593788 11121 consensus_queue.cc:1035] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:09.594205 11121 consensus_queue.cc:1035] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:09.607949 11169 mvcc.cc:205] Tried to move back new op lower bound from 7023241091426881536 to 7023241089918668800. Current Snapshot: MvccSnapshot[applied={T|T < 7023241091426881536}]
I20240502 14:01:09.608829 11163 mvcc.cc:205] Tried to move back new op lower bound from 7023241091411742720 to 7023241090402979840. Current Snapshot: MvccSnapshot[applied={T|T < 7023241091389386752}]
I20240502 14:01:09.610611 11132 mvcc.cc:205] Tried to move back new op lower bound from 7023241091426881536 to 7023241089918668800. Current Snapshot: MvccSnapshot[applied={T|T < 7023241091426881536}]
INSERT report
rows total: 2000
time total: 106.828 ms
time per row: 0.0534142 ms
I20240502 14:01:09.647363 9968 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44294:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:01:09.653196 10356 tablet_service.cc:1467] Processing CreateTablet for tablet 0b80a9c354354433a48a0a5f8938265d (DEFAULT_TABLE table=rebalance_test_table_1 [id=4111fcf0def5440c9d789797606f1b0b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:09.653275 10225 tablet_service.cc:1467] Processing CreateTablet for tablet 0b80a9c354354433a48a0a5f8938265d (DEFAULT_TABLE table=rebalance_test_table_1 [id=4111fcf0def5440c9d789797606f1b0b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:09.653318 10487 tablet_service.cc:1467] Processing CreateTablet for tablet 0b80a9c354354433a48a0a5f8938265d (DEFAULT_TABLE table=rebalance_test_table_1 [id=4111fcf0def5440c9d789797606f1b0b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:09.653544 10356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b80a9c354354433a48a0a5f8938265d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:09.653584 10225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b80a9c354354433a48a0a5f8938265d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:09.653574 10487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b80a9c354354433a48a0a5f8938265d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:09.654470 10355 tablet_service.cc:1467] Processing CreateTablet for tablet a804b61491304055865347e70e8a9778 (DEFAULT_TABLE table=rebalance_test_table_1 [id=4111fcf0def5440c9d789797606f1b0b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:09.654704 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a804b61491304055865347e70e8a9778. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:09.656018 10486 tablet_service.cc:1467] Processing CreateTablet for tablet a804b61491304055865347e70e8a9778 (DEFAULT_TABLE table=rebalance_test_table_1 [id=4111fcf0def5440c9d789797606f1b0b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:09.656263 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a804b61491304055865347e70e8a9778. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:09.656584 10354 tablet_service.cc:1467] Processing CreateTablet for tablet f0b83b1163014375a53d97d95f4fcb13 (DEFAULT_TABLE table=rebalance_test_table_1 [id=4111fcf0def5440c9d789797606f1b0b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:09.656816 10354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0b83b1163014375a53d97d95f4fcb13. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:09.656999 11190 tablet_bootstrap.cc:492] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:09.657037 10092 tablet_service.cc:1467] Processing CreateTablet for tablet f0b83b1163014375a53d97d95f4fcb13 (DEFAULT_TABLE table=rebalance_test_table_1 [id=4111fcf0def5440c9d789797606f1b0b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:09.657315 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0b83b1163014375a53d97d95f4fcb13. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:09.658027 11190 tablet_bootstrap.cc:654] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:09.658790 10224 tablet_service.cc:1467] Processing CreateTablet for tablet a804b61491304055865347e70e8a9778 (DEFAULT_TABLE table=rebalance_test_table_1 [id=4111fcf0def5440c9d789797606f1b0b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:09.659035 10224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a804b61491304055865347e70e8a9778. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:09.659135 10223 tablet_service.cc:1467] Processing CreateTablet for tablet f0b83b1163014375a53d97d95f4fcb13 (DEFAULT_TABLE table=rebalance_test_table_1 [id=4111fcf0def5440c9d789797606f1b0b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:09.659382 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0b83b1163014375a53d97d95f4fcb13. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:09.659888 11192 tablet_bootstrap.cc:492] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:09.660362 11189 tablet_bootstrap.cc:492] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:09.660928 11192 tablet_bootstrap.cc:654] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:09.661514 11188 tablet_bootstrap.cc:492] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:09.661542 11190 tablet_bootstrap.cc:492] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:09.661643 11190 ts_tablet_manager.cc:1367] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:01:09.662150 11190 raft_consensus.cc:348] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:09.662230 11192 tablet_bootstrap.cc:492] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:09.662266 11190 raft_consensus.cc:374] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:09.662300 11192 ts_tablet_manager.cc:1367] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:09.662310 11190 raft_consensus.cc:729] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:09.662472 11190 consensus_queue.cc:260] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:09.662765 11192 raft_consensus.cc:348] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.662878 11192 raft_consensus.cc:374] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:09.662918 11192 raft_consensus.cc:729] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:09.663039 11192 consensus_queue.cc:260] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.663218 11188 tablet_bootstrap.cc:654] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:09.663322 11192 ts_tablet_manager.cc:1398] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:09.663508 11189 tablet_bootstrap.cc:654] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:09.664283 11190 ts_tablet_manager.cc:1398] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:01:09.664471 11190 tablet_bootstrap.cc:492] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:09.665395 11190 tablet_bootstrap.cc:654] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:09.666028 11189 tablet_bootstrap.cc:492] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:09.666101 11189 ts_tablet_manager.cc:1367] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:01:09.666512 11190 tablet_bootstrap.cc:492] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:09.666474 11189 raft_consensus.cc:348] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:09.666559 11189 raft_consensus.cc:374] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:09.666586 11189 raft_consensus.cc:729] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:09.666586 11190 ts_tablet_manager.cc:1367] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:09.666815 11189 consensus_queue.cc:260] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:09.666905 11190 raft_consensus.cc:348] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:09.666989 11190 raft_consensus.cc:374] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:09.667024 11190 raft_consensus.cc:729] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:09.667148 11189 ts_tablet_manager.cc:1398] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:09.667145 11190 consensus_queue.cc:260] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:09.667384 11190 ts_tablet_manager.cc:1398] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:09.667572 11190 tablet_bootstrap.cc:492] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:09.667598 11188 tablet_bootstrap.cc:492] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:09.667675 11188 ts_tablet_manager.cc:1367] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:01:09.668267 11188 raft_consensus.cc:348] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:09.668388 11188 raft_consensus.cc:374] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:09.668424 11188 raft_consensus.cc:729] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:09.668463 11190 tablet_bootstrap.cc:654] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:09.668574 11188 consensus_queue.cc:260] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:09.668948 11188 ts_tablet_manager.cc:1398] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:09.669131 11188 tablet_bootstrap.cc:492] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:09.669461 11189 tablet_bootstrap.cc:492] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:09.670008 11188 tablet_bootstrap.cc:654] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:09.670213 11190 tablet_bootstrap.cc:492] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:09.670282 11190 ts_tablet_manager.cc:1367] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:09.670367 11189 tablet_bootstrap.cc:654] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:09.670746 11190 raft_consensus.cc:348] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.670850 11190 raft_consensus.cc:374] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:09.670881 11190 raft_consensus.cc:729] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:09.670984 11190 consensus_queue.cc:260] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.671224 11190 ts_tablet_manager.cc:1398] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:09.671352 11189 tablet_bootstrap.cc:492] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:09.671435 11189 ts_tablet_manager.cc:1367] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:09.671860 11189 raft_consensus.cc:348] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:09.671957 11189 raft_consensus.cc:374] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:09.671984 11189 raft_consensus.cc:729] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:09.672099 11189 consensus_queue.cc:260] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:09.672317 11189 ts_tablet_manager.cc:1398] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:09.673058 11188 tablet_bootstrap.cc:492] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:09.673127 11188 ts_tablet_manager.cc:1367] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:01:09.673489 11188 raft_consensus.cc:348] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:09.673574 11188 raft_consensus.cc:374] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:09.673602 11188 raft_consensus.cc:729] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:09.673723 11188 consensus_queue.cc:260] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:09.674002 11188 ts_tablet_manager.cc:1398] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:09.674129 11188 tablet_bootstrap.cc:492] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:09.674988 11188 tablet_bootstrap.cc:654] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:09.676077 11188 tablet_bootstrap.cc:492] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:09.676172 11188 ts_tablet_manager.cc:1367] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:09.676638 11188 raft_consensus.cc:348] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.676743 11188 raft_consensus.cc:374] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:09.676800 11188 raft_consensus.cc:729] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:09.676926 11188 consensus_queue.cc:260] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.677188 11188 ts_tablet_manager.cc:1398] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:09.684444 11117 raft_consensus.cc:482] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:09.684545 11117 raft_consensus.cc:504] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.684907 11117 leader_election.cc:288] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:09.685233 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b83b1163014375a53d97d95f4fcb13" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:09.685251 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b83b1163014375a53d97d95f4fcb13" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:09.685355 10375 raft_consensus.cc:2454] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:09.685386 10112 raft_consensus.cc:2454] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:09.685618 10178 leader_election.cc:302] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:09.685776 11117 raft_consensus.cc:2781] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:09.685835 11117 raft_consensus.cc:482] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:09.685878 11117 raft_consensus.cc:3037] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.686654 11117 raft_consensus.cc:504] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.686920 11117 leader_election.cc:288] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:09.687235 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b83b1163014375a53d97d95f4fcb13" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:09.687340 10375 raft_consensus.cc:3037] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.687970 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b83b1163014375a53d97d95f4fcb13" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:09.688107 10112 raft_consensus.cc:3037] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.688215 10375 raft_consensus.cc:2454] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:09.688486 10178 leader_election.cc:302] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:09.688970 10112 raft_consensus.cc:2454] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:09.689162 11117 raft_consensus.cc:2781] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:09.689252 11117 raft_consensus.cc:686] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:09.689390 11117 consensus_queue.cc:237] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:09.690910 9968 catalog_manager.cc:5484] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 1 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:09.808507 11117 raft_consensus.cc:482] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:09.808653 11117 raft_consensus.cc:504] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:09.809139 11117 leader_election.cc:288] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:09.809631 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a804b61491304055865347e70e8a9778" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:09.809650 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a804b61491304055865347e70e8a9778" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:09.809791 10375 raft_consensus.cc:2454] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:09.809791 10507 raft_consensus.cc:2454] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:09.810089 10180 leader_election.cc:302] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:09.810271 11117 raft_consensus.cc:2781] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:09.810315 11117 raft_consensus.cc:482] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:09.810343 11117 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.811182 11117 raft_consensus.cc:504] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:09.811507 11117 leader_election.cc:288] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:09.811808 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a804b61491304055865347e70e8a9778" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:09.811864 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a804b61491304055865347e70e8a9778" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:09.811957 10375 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.812004 10507 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.812714 10375 raft_consensus.cc:2454] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:09.812754 10507 raft_consensus.cc:2454] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:09.813027 10180 leader_election.cc:302] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:09.813210 11117 raft_consensus.cc:2781] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:09.813294 11117 raft_consensus.cc:686] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:09.813434 11117 consensus_queue.cc:237] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:09.815048 9968 catalog_manager.cc:5484] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 1 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:09.911660 11121 raft_consensus.cc:482] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:09.911846 11121 raft_consensus.cc:504] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:09.912376 11121 leader_election.cc:288] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:09.913213 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:09.913363 10507 raft_consensus.cc:2454] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:09.913719 10311 leader_election.cc:302] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:09.913897 11121 raft_consensus.cc:2781] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:09.913960 11121 raft_consensus.cc:482] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:09.914000 11121 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.915030 11121 raft_consensus.cc:504] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:09.915359 11121 leader_election.cc:288] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:09.915699 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:09.915849 10507 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:09.916554 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:09.916617 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:09.916750 10507 raft_consensus.cc:2454] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:09.916759 10245 raft_consensus.cc:2454] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:09.917191 10311 leader_election.cc:302] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters: 1ca5103abf454aaca375e9e07eb139cc
I20240502 14:01:09.917397 11121 raft_consensus.cc:2781] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:09.917490 11121 raft_consensus.cc:686] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:09.917631 11121 consensus_queue.cc:237] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:09.919273 9968 catalog_manager.cc:5484] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:10.065694 11205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:10.065956 11205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:10.069140 11205 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 14:01:10.147768 10507 raft_consensus.cc:1261] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.147951 10244 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.148312 11111 consensus_queue.cc:1035] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:10.149138 10244 raft_consensus.cc:1261] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.149828 10112 raft_consensus.cc:1261] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.150259 11108 consensus_queue.cc:1035] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:10.151065 10508 raft_consensus.cc:1261] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.151784 11111 consensus_queue.cc:1035] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:10.152232 10375 raft_consensus.cc:1261] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.152479 10376 raft_consensus.cc:1261] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.152666 11108 consensus_queue.cc:1035] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:10.153558 11108 consensus_queue.cc:1035] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:10.153877 11108 consensus_queue.cc:1035] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:10.167168 11248 mvcc.cc:205] Tried to move back new op lower bound from 7023241093721014272 to 7023241092783812608. Current Snapshot: MvccSnapshot[applied={T|T < 7023241093721014272 or (T in {7023241093721014272})}]
INSERT report
rows total: 2000
time total: 97.3774 ms
time per row: 0.0486887 ms
I20240502 14:01:10.212164 9972 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44294:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:01:10.217478 10092 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec3efe6d5bc45a79458230c43f26b49 (DEFAULT_TABLE table=rebalance_test_table_2 [id=24d9b1d14e9f47609493e9203ef93ab3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.217518 10223 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec3efe6d5bc45a79458230c43f26b49 (DEFAULT_TABLE table=rebalance_test_table_2 [id=24d9b1d14e9f47609493e9203ef93ab3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.217792 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec3efe6d5bc45a79458230c43f26b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.217792 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec3efe6d5bc45a79458230c43f26b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.217926 10354 tablet_service.cc:1467] Processing CreateTablet for tablet 45f819d4fb5946248f5ec084d252bcd0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=24d9b1d14e9f47609493e9203ef93ab3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.218168 10354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45f819d4fb5946248f5ec084d252bcd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.219786 11262 tablet_bootstrap.cc:492] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:10.220134 10486 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec3efe6d5bc45a79458230c43f26b49 (DEFAULT_TABLE table=rebalance_test_table_2 [id=24d9b1d14e9f47609493e9203ef93ab3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.220415 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec3efe6d5bc45a79458230c43f26b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.220513 10487 tablet_service.cc:1467] Processing CreateTablet for tablet 45f819d4fb5946248f5ec084d252bcd0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=24d9b1d14e9f47609493e9203ef93ab3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.220548 10485 tablet_service.cc:1467] Processing CreateTablet for tablet 3474ecec1b6f4fc29d6db00b8fa56b06 (DEFAULT_TABLE table=rebalance_test_table_2 [id=24d9b1d14e9f47609493e9203ef93ab3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.220719 10487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45f819d4fb5946248f5ec084d252bcd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.220778 11262 tablet_bootstrap.cc:654] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.221100 10485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3474ecec1b6f4fc29d6db00b8fa56b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.221235 10093 tablet_service.cc:1467] Processing CreateTablet for tablet 3474ecec1b6f4fc29d6db00b8fa56b06 (DEFAULT_TABLE table=rebalance_test_table_2 [id=24d9b1d14e9f47609493e9203ef93ab3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.221475 10093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3474ecec1b6f4fc29d6db00b8fa56b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.222715 10092 tablet_service.cc:1467] Processing CreateTablet for tablet 45f819d4fb5946248f5ec084d252bcd0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=24d9b1d14e9f47609493e9203ef93ab3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.222951 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45f819d4fb5946248f5ec084d252bcd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.223609 10223 tablet_service.cc:1467] Processing CreateTablet for tablet 3474ecec1b6f4fc29d6db00b8fa56b06 (DEFAULT_TABLE table=rebalance_test_table_2 [id=24d9b1d14e9f47609493e9203ef93ab3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.223871 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3474ecec1b6f4fc29d6db00b8fa56b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.224299 11265 tablet_bootstrap.cc:492] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:10.224406 11262 tablet_bootstrap.cc:492] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:10.224478 11262 ts_tablet_manager.cc:1367] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:01:10.224932 11263 tablet_bootstrap.cc:492] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:10.224913 11262 raft_consensus.cc:348] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.225054 11262 raft_consensus.cc:374] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.225102 11262 raft_consensus.cc:729] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.225132 11265 tablet_bootstrap.cc:654] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.225244 11262 consensus_queue.cc:260] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.225337 11264 tablet_bootstrap.cc:492] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:10.225584 11262 ts_tablet_manager.cc:1398] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.226802 11265 tablet_bootstrap.cc:492] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:10.226902 11265 ts_tablet_manager.cc:1367] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:10.226949 11263 tablet_bootstrap.cc:654] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.227274 11264 tablet_bootstrap.cc:654] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.227277 11265 raft_consensus.cc:348] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.227407 11265 raft_consensus.cc:374] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.227461 11265 raft_consensus.cc:729] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.227594 11265 consensus_queue.cc:260] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.227900 11265 ts_tablet_manager.cc:1398] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.228063 11265 tablet_bootstrap.cc:492] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:10.228935 11264 tablet_bootstrap.cc:492] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:10.228974 11265 tablet_bootstrap.cc:654] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.229084 11264 ts_tablet_manager.cc:1367] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 14:01:10.229514 11264 raft_consensus.cc:348] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.229667 11264 raft_consensus.cc:374] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.229734 11264 raft_consensus.cc:729] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.229921 11264 consensus_queue.cc:260] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.230187 11265 tablet_bootstrap.cc:492] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:10.230269 11265 ts_tablet_manager.cc:1367] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:10.230396 11264 ts_tablet_manager.cc:1398] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:10.230579 11264 tablet_bootstrap.cc:492] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:10.230726 11265 raft_consensus.cc:348] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.230849 11265 raft_consensus.cc:374] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.230901 11265 raft_consensus.cc:729] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.231040 11265 consensus_queue.cc:260] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.231300 11265 ts_tablet_manager.cc:1398] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.231616 11265 tablet_bootstrap.cc:492] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:10.232426 11265 tablet_bootstrap.cc:654] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.233283 11263 tablet_bootstrap.cc:492] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:10.233409 11263 ts_tablet_manager.cc:1367] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:01:10.233927 11265 tablet_bootstrap.cc:492] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:10.234041 11265 ts_tablet_manager.cc:1367] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:10.234442 11265 raft_consensus.cc:348] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.234588 11265 raft_consensus.cc:374] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.234519 11263 raft_consensus.cc:348] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.234639 11263 raft_consensus.cc:374] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.234669 11265 raft_consensus.cc:729] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.234694 11263 raft_consensus.cc:729] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.234889 11265 consensus_queue.cc:260] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.234910 11263 consensus_queue.cc:260] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.235256 11263 ts_tablet_manager.cc:1398] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:10.235431 11263 tablet_bootstrap.cc:492] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:10.235813 11265 ts_tablet_manager.cc:1398] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:10.236310 11263 tablet_bootstrap.cc:654] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.237622 11264 tablet_bootstrap.cc:654] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.238385 11263 tablet_bootstrap.cc:492] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:10.238467 11263 ts_tablet_manager.cc:1367] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:10.238857 11263 raft_consensus.cc:348] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.238965 11263 raft_consensus.cc:374] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.238991 11263 raft_consensus.cc:729] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.239085 11263 consensus_queue.cc:260] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.239570 11263 ts_tablet_manager.cc:1398] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.239744 11263 tablet_bootstrap.cc:492] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:10.240032 11264 tablet_bootstrap.cc:492] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:10.240121 11264 ts_tablet_manager.cc:1367] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240502 14:01:10.240595 11264 raft_consensus.cc:348] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.240717 11264 raft_consensus.cc:374] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.240751 11263 tablet_bootstrap.cc:654] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.240764 11264 raft_consensus.cc:729] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.240907 11264 consensus_queue.cc:260] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.241194 11264 ts_tablet_manager.cc:1398] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.242100 11263 tablet_bootstrap.cc:492] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:10.242188 11263 ts_tablet_manager.cc:1367] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:10.242671 11263 raft_consensus.cc:348] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.242821 11263 raft_consensus.cc:374] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.242866 11263 raft_consensus.cc:729] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.243003 11263 consensus_queue.cc:260] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.243338 11263 ts_tablet_manager.cc:1398] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.264921 11125 raft_consensus.cc:482] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:10.265072 11125 raft_consensus.cc:504] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.266150 11125 leader_election.cc:288] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:10.270900 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f819d4fb5946248f5ec084d252bcd0" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:10.271041 10508 raft_consensus.cc:2454] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:10.271291 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f819d4fb5946248f5ec084d252bcd0" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:10.271436 10376 raft_consensus.cc:2454] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:10.271399 10049 leader_election.cc:302] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:10.271641 11125 raft_consensus.cc:2781] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:10.271700 11125 raft_consensus.cc:482] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:10.271752 11125 raft_consensus.cc:3037] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.272590 11125 raft_consensus.cc:504] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.272951 11125 leader_election.cc:288] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:10.273221 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f819d4fb5946248f5ec084d252bcd0" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:10.273305 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f819d4fb5946248f5ec084d252bcd0" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:10.273350 10508 raft_consensus.cc:3037] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.273423 10376 raft_consensus.cc:3037] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.274127 10508 raft_consensus.cc:2454] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:10.274139 10376 raft_consensus.cc:2454] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:10.274399 10049 leader_election.cc:302] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:10.274554 11125 raft_consensus.cc:2781] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:10.274760 11125 raft_consensus.cc:686] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Leader. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:10.275070 11125 consensus_queue.cc:237] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.277189 9968 catalog_manager.cc:5484] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be reported cstate change: term changed from 0 to 1, leader changed from <none> to 068f58913ea3478599df829d5118f7be (127.26.25.129). New cstate: current_term: 1 leader_uuid: "068f58913ea3478599df829d5118f7be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:10.342479 11117 raft_consensus.cc:482] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:10.342617 11117 raft_consensus.cc:504] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.342986 11117 leader_election.cc:288] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:10.343490 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:10.343490 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:10.343650 10508 raft_consensus.cc:2454] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:10.343650 10112 raft_consensus.cc:2454] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:10.343971 10180 leader_election.cc:302] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:10.344146 11117 raft_consensus.cc:2781] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:10.344208 11117 raft_consensus.cc:482] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:10.344269 11117 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.345050 11117 raft_consensus.cc:504] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.345353 11117 leader_election.cc:288] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:10.345773 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:10.345774 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:10.345898 10508 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.345919 10112 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.346818 10112 raft_consensus.cc:2454] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:10.346827 10508 raft_consensus.cc:2454] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:10.347157 10180 leader_election.cc:302] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:10.347334 11117 raft_consensus.cc:2781] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:10.347429 11117 raft_consensus.cc:686] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:10.347568 11117 consensus_queue.cc:237] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.349273 9968 catalog_manager.cc:5484] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 1 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:10.470263 11130 raft_consensus.cc:482] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:10.470407 11130 raft_consensus.cc:504] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.471477 11130 leader_election.cc:288] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:10.475131 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3474ecec1b6f4fc29d6db00b8fa56b06" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:10.475174 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3474ecec1b6f4fc29d6db00b8fa56b06" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:10.475301 10112 raft_consensus.cc:2454] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 0.
I20240502 14:01:10.475301 10244 raft_consensus.cc:2454] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 0.
I20240502 14:01:10.475651 10442 leader_election.cc:302] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:10.475903 11130 raft_consensus.cc:2781] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:10.475970 11130 raft_consensus.cc:482] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:10.476014 11130 raft_consensus.cc:3037] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.476855 11130 raft_consensus.cc:504] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.477159 11130 leader_election.cc:288] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:10.477523 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3474ecec1b6f4fc29d6db00b8fa56b06" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:10.477536 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3474ecec1b6f4fc29d6db00b8fa56b06" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:10.477648 10244 raft_consensus.cc:3037] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.477648 10112 raft_consensus.cc:3037] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.478590 10112 raft_consensus.cc:2454] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 1.
I20240502 14:01:10.478590 10244 raft_consensus.cc:2454] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 1.
I20240502 14:01:10.478888 10442 leader_election.cc:302] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:10.479084 11130 raft_consensus.cc:2781] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:10.479265 11130 raft_consensus.cc:686] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:10.479538 11130 consensus_queue.cc:237] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:10.481632 9968 catalog_manager.cc:5484] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } }
W20240502 14:01:10.632508 11286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:10.632803 11286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:10.635915 11286 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 14:01:10.724532 10376 raft_consensus.cc:1261] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.725257 10508 raft_consensus.cc:1261] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.725654 11278 consensus_queue.cc:1035] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:10.725891 11125 consensus_queue.cc:1035] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:10.728932 10112 raft_consensus.cc:1261] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.728928 10244 raft_consensus.cc:1261] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.729339 11130 consensus_queue.cc:1035] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:10.729488 11284 consensus_queue.cc:1035] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:10.733549 10507 raft_consensus.cc:1261] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.734194 10113 raft_consensus.cc:1261] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:10.734340 11117 consensus_queue.cc:1035] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:10.734825 11117 consensus_queue.cc:1035] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 121.163 ms
time per row: 0.0605817 ms
I20240502 14:01:10.799139 9966 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44294:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:01:10.804750 10092 tablet_service.cc:1467] Processing CreateTablet for tablet d9325aa99d6d48d2bcbc39ffdce77bf1 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f2432e4987d644468d2ed387a8e8a5bc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.804811 10223 tablet_service.cc:1467] Processing CreateTablet for tablet d9325aa99d6d48d2bcbc39ffdce77bf1 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f2432e4987d644468d2ed387a8e8a5bc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.805106 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9325aa99d6d48d2bcbc39ffdce77bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.805128 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9325aa99d6d48d2bcbc39ffdce77bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.805264 10354 tablet_service.cc:1467] Processing CreateTablet for tablet d9325aa99d6d48d2bcbc39ffdce77bf1 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f2432e4987d644468d2ed387a8e8a5bc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.805549 10354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9325aa99d6d48d2bcbc39ffdce77bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.807024 11342 tablet_bootstrap.cc:492] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:10.807704 10355 tablet_service.cc:1467] Processing CreateTablet for tablet 8b1dff0b06824632869c970e3a031d3b (DEFAULT_TABLE table=rebalance_test_table_3 [id=f2432e4987d644468d2ed387a8e8a5bc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.807754 10356 tablet_service.cc:1467] Processing CreateTablet for tablet ef18a1e73625480a94c556f7960255a7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f2432e4987d644468d2ed387a8e8a5bc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.807957 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b1dff0b06824632869c970e3a031d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.808166 10356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef18a1e73625480a94c556f7960255a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.808403 11344 tablet_bootstrap.cc:492] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:10.808671 10486 tablet_service.cc:1467] Processing CreateTablet for tablet 8b1dff0b06824632869c970e3a031d3b (DEFAULT_TABLE table=rebalance_test_table_3 [id=f2432e4987d644468d2ed387a8e8a5bc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.808162 11342 tablet_bootstrap.cc:654] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.808939 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b1dff0b06824632869c970e3a031d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.809211 11344 tablet_bootstrap.cc:654] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.809832 11343 tablet_bootstrap.cc:492] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:10.810252 11344 tablet_bootstrap.cc:492] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:10.810325 11344 ts_tablet_manager.cc:1367] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:10.810717 11343 tablet_bootstrap.cc:654] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.810827 11344 raft_consensus.cc:348] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.810946 11344 raft_consensus.cc:374] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.810981 11344 raft_consensus.cc:729] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.811123 11344 consensus_queue.cc:260] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.811133 10485 tablet_service.cc:1467] Processing CreateTablet for tablet ef18a1e73625480a94c556f7960255a7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f2432e4987d644468d2ed387a8e8a5bc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.811343 10485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef18a1e73625480a94c556f7960255a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.812050 10223 tablet_service.cc:1467] Processing CreateTablet for tablet ef18a1e73625480a94c556f7960255a7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f2432e4987d644468d2ed387a8e8a5bc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.812181 11344 ts_tablet_manager.cc:1398] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:10.812268 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef18a1e73625480a94c556f7960255a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.812358 11344 tablet_bootstrap.cc:492] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:10.813148 11344 tablet_bootstrap.cc:654] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.813489 10092 tablet_service.cc:1467] Processing CreateTablet for tablet 8b1dff0b06824632869c970e3a031d3b (DEFAULT_TABLE table=rebalance_test_table_3 [id=f2432e4987d644468d2ed387a8e8a5bc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:10.813684 11347 tablet_bootstrap.cc:492] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:10.813737 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b1dff0b06824632869c970e3a031d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:10.814510 11342 tablet_bootstrap.cc:492] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:10.814608 11342 ts_tablet_manager.cc:1367] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:01:10.815125 11342 raft_consensus.cc:348] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.815238 11342 raft_consensus.cc:374] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.815281 11342 raft_consensus.cc:729] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.815411 11342 consensus_queue.cc:260] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.815709 11342 ts_tablet_manager.cc:1398] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.815908 11342 tablet_bootstrap.cc:492] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:10.816648 11347 tablet_bootstrap.cc:654] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.816812 11342 tablet_bootstrap.cc:654] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.817018 11344 tablet_bootstrap.cc:492] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:10.817106 11344 ts_tablet_manager.cc:1367] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:01:10.817519 11344 raft_consensus.cc:348] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:10.817634 11344 raft_consensus.cc:374] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.817672 11344 raft_consensus.cc:729] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.817821 11344 consensus_queue.cc:260] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:10.818094 11343 tablet_bootstrap.cc:492] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:10.818171 11343 ts_tablet_manager.cc:1367] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:01:10.818171 11344 ts_tablet_manager.cc:1398] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.818362 11344 tablet_bootstrap.cc:492] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:10.818629 11343 raft_consensus.cc:348] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.818742 11343 raft_consensus.cc:374] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.818783 11343 raft_consensus.cc:729] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.818907 11343 consensus_queue.cc:260] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.819156 11343 ts_tablet_manager.cc:1398] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.819217 11344 tablet_bootstrap.cc:654] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.819319 11343 tablet_bootstrap.cc:492] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:10.820210 11343 tablet_bootstrap.cc:654] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.820248 11344 tablet_bootstrap.cc:492] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:10.820325 11344 ts_tablet_manager.cc:1367] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:10.820739 11344 raft_consensus.cc:348] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.820837 11344 raft_consensus.cc:374] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.820868 11344 raft_consensus.cc:729] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.820984 11344 consensus_queue.cc:260] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.821311 11344 ts_tablet_manager.cc:1398] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.821498 11347 tablet_bootstrap.cc:492] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:10.821633 11347 ts_tablet_manager.cc:1367] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:01:10.822168 11347 raft_consensus.cc:348] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.822311 11347 raft_consensus.cc:374] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.822376 11347 raft_consensus.cc:729] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.822544 11347 consensus_queue.cc:260] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.822822 11347 ts_tablet_manager.cc:1398] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.822990 11347 tablet_bootstrap.cc:492] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:10.823529 11342 tablet_bootstrap.cc:492] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:10.823627 11342 ts_tablet_manager.cc:1367] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:01:10.823896 11347 tablet_bootstrap.cc:654] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:10.824154 11342 raft_consensus.cc:348] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.824254 11342 raft_consensus.cc:374] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.824285 11342 raft_consensus.cc:729] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.824457 11343 tablet_bootstrap.cc:492] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:10.824424 11342 consensus_queue.cc:260] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.824527 11343 ts_tablet_manager.cc:1367] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:01:10.824954 11343 raft_consensus.cc:348] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:10.825066 11343 raft_consensus.cc:374] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.825100 11343 raft_consensus.cc:729] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.825143 11347 tablet_bootstrap.cc:492] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:10.825219 11347 ts_tablet_manager.cc:1367] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:10.825215 11343 consensus_queue.cc:260] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:10.825495 11343 ts_tablet_manager.cc:1398] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:10.825651 11347 raft_consensus.cc:348] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:10.825747 11347 raft_consensus.cc:374] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:10.825776 11347 raft_consensus.cc:729] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:10.825878 11347 consensus_queue.cc:260] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:10.826041 11342 ts_tablet_manager.cc:1398] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:10.826134 11347 ts_tablet_manager.cc:1398] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:10.904150 11247 raft_consensus.cc:482] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:10.904310 11247 raft_consensus.cc:504] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:10.904747 11247 leader_election.cc:288] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:10.905241 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:10.905264 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:10.905383 10244 raft_consensus.cc:2454] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:10.905414 10507 raft_consensus.cc:2454] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:10.905671 10311 leader_election.cc:302] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:10.905828 11247 raft_consensus.cc:2781] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:10.905889 11247 raft_consensus.cc:482] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:10.905943 11247 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.906786 11247 raft_consensus.cc:504] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:10.907131 11247 leader_election.cc:288] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:10.907465 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:10.907505 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:10.907588 10507 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.907591 10244 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.908552 10244 raft_consensus.cc:2454] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:10.908552 10507 raft_consensus.cc:2454] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:10.908818 10311 leader_election.cc:302] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:10.909047 11247 raft_consensus.cc:2781] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:10.909250 11247 raft_consensus.cc:686] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:10.909409 11247 consensus_queue.cc:237] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:10.910982 9964 catalog_manager.cc:5484] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:10.947376 11130 raft_consensus.cc:482] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:10.947561 11130 raft_consensus.cc:504] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.948153 11130 leader_election.cc:288] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:10.948604 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b1dff0b06824632869c970e3a031d3b" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:10.948789 10113 raft_consensus.cc:2454] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 0.
I20240502 14:01:10.949132 10442 leader_election.cc:302] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:10.949333 11130 raft_consensus.cc:2781] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:10.949399 11130 raft_consensus.cc:482] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:10.949437 11130 raft_consensus.cc:3037] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.950515 11130 raft_consensus.cc:504] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.950899 11130 leader_election.cc:288] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:10.951295 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b1dff0b06824632869c970e3a031d3b" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:10.951412 10113 raft_consensus.cc:3037] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.952548 10113 raft_consensus.cc:2454] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 1.
I20240502 14:01:10.953075 10442 leader_election.cc:302] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:10.953073 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b1dff0b06824632869c970e3a031d3b" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:10.953064 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b1dff0b06824632869c970e3a031d3b" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:10.953217 10375 raft_consensus.cc:3037] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.953254 11130 raft_consensus.cc:2781] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:10.953326 11130 raft_consensus.cc:686] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:10.953481 11130 consensus_queue.cc:237] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.954104 10375 raft_consensus.cc:2454] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 1.
I20240502 14:01:10.955458 9964 catalog_manager.cc:5484] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:10.958632 11278 raft_consensus.cc:482] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:10.958730 11278 raft_consensus.cc:504] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.959133 11278 leader_election.cc:288] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:10.959489 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:10.959623 10375 raft_consensus.cc:2454] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:10.959945 10047 leader_election.cc:302] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:10.960109 11278 raft_consensus.cc:2781] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:10.960160 11278 raft_consensus.cc:482] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:10.960187 11278 raft_consensus.cc:3037] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.960960 11278 raft_consensus.cc:504] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.961263 11278 leader_election.cc:288] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:10.961899 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:10.962010 10375 raft_consensus.cc:3037] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:10.962714 10375 raft_consensus.cc:2454] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:10.962864 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:10.962922 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:10.963003 10244 raft_consensus.cc:2454] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:10.963016 10047 leader_election.cc:302] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:10.963217 11278 raft_consensus.cc:2781] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:10.963307 11278 raft_consensus.cc:686] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Leader. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:10.963469 11278 consensus_queue.cc:237] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:10.965128 9964 catalog_manager.cc:5484] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be reported cstate change: term changed from 0 to 1, leader changed from <none> to 068f58913ea3478599df829d5118f7be (127.26.25.129). New cstate: current_term: 1 leader_uuid: "068f58913ea3478599df829d5118f7be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:11.110191 11361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:11.110451 11361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:11.113546 11361 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 14:01:11.197270 10245 raft_consensus.cc:3037] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:11.198284 10245 raft_consensus.cc:1261] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:11.200285 10375 raft_consensus.cc:1261] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:11.200707 11278 consensus_queue.cc:1035] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:11.200964 10507 raft_consensus.cc:1261] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:11.200942 11125 consensus_queue.cc:1035] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:11.201367 11355 consensus_queue.cc:1035] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:11.201797 10244 raft_consensus.cc:1261] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:11.207482 10375 raft_consensus.cc:1261] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:11.207502 10113 raft_consensus.cc:1261] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:11.207561 11355 consensus_queue.cc:1035] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:11.208106 11130 consensus_queue.cc:1035] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:11.208425 11130 consensus_queue.cc:1035] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 114.973 ms
time per row: 0.0574865 ms
I20240502 14:01:11.272339 9971 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44294:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:01:11.277674 10223 tablet_service.cc:1467] Processing CreateTablet for tablet 657170da057f4318a6165d2ed9097e58 (DEFAULT_TABLE table=rebalance_test_table_4 [id=bcd4762c4e704dd7b12bcc6533269ce9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.277671 10092 tablet_service.cc:1467] Processing CreateTablet for tablet 657170da057f4318a6165d2ed9097e58 (DEFAULT_TABLE table=rebalance_test_table_4 [id=bcd4762c4e704dd7b12bcc6533269ce9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.278012 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 657170da057f4318a6165d2ed9097e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.278014 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 657170da057f4318a6165d2ed9097e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.278055 10224 tablet_service.cc:1467] Processing CreateTablet for tablet aeceb15b637841609765e29e9b4d7ef1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=bcd4762c4e704dd7b12bcc6533269ce9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.278254 10224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeceb15b637841609765e29e9b4d7ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.278831 10486 tablet_service.cc:1467] Processing CreateTablet for tablet 67cf72901dd0434d9b88902343d45977 (DEFAULT_TABLE table=rebalance_test_table_4 [id=bcd4762c4e704dd7b12bcc6533269ce9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.279114 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67cf72901dd0434d9b88902343d45977. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.280332 10093 tablet_service.cc:1467] Processing CreateTablet for tablet aeceb15b637841609765e29e9b4d7ef1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=bcd4762c4e704dd7b12bcc6533269ce9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.280355 10485 tablet_service.cc:1467] Processing CreateTablet for tablet aeceb15b637841609765e29e9b4d7ef1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=bcd4762c4e704dd7b12bcc6533269ce9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.280558 10093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeceb15b637841609765e29e9b4d7ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.280558 10485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeceb15b637841609765e29e9b4d7ef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.281312 10094 tablet_service.cc:1467] Processing CreateTablet for tablet 67cf72901dd0434d9b88902343d45977 (DEFAULT_TABLE table=rebalance_test_table_4 [id=bcd4762c4e704dd7b12bcc6533269ce9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.281492 10094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67cf72901dd0434d9b88902343d45977. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.282474 11343 tablet_bootstrap.cc:492] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:11.282486 11342 tablet_bootstrap.cc:492] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:11.282985 11347 tablet_bootstrap.cc:492] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:11.283309 11342 tablet_bootstrap.cc:654] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.283344 11343 tablet_bootstrap.cc:654] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.283349 10355 tablet_service.cc:1467] Processing CreateTablet for tablet 657170da057f4318a6165d2ed9097e58 (DEFAULT_TABLE table=rebalance_test_table_4 [id=bcd4762c4e704dd7b12bcc6533269ce9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.283632 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 657170da057f4318a6165d2ed9097e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.283807 11347 tablet_bootstrap.cc:654] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.284456 11342 tablet_bootstrap.cc:492] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:11.284533 11342 ts_tablet_manager.cc:1367] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:11.285032 11342 raft_consensus.cc:348] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.285142 11342 raft_consensus.cc:374] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.285179 11342 raft_consensus.cc:729] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.285286 11347 tablet_bootstrap.cc:492] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:11.285301 11342 consensus_queue.cc:260] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.285356 11347 ts_tablet_manager.cc:1367] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:01:11.285578 11342 ts_tablet_manager.cc:1398] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:11.285748 11342 tablet_bootstrap.cc:492] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:11.285800 11347 raft_consensus.cc:348] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.285889 11347 raft_consensus.cc:374] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.285919 11347 raft_consensus.cc:729] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.286034 11347 consensus_queue.cc:260] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.286180 11344 tablet_bootstrap.cc:492] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:11.286211 10356 tablet_service.cc:1467] Processing CreateTablet for tablet 67cf72901dd0434d9b88902343d45977 (DEFAULT_TABLE table=rebalance_test_table_4 [id=bcd4762c4e704dd7b12bcc6533269ce9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.286325 11347 ts_tablet_manager.cc:1398] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:11.286410 10356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67cf72901dd0434d9b88902343d45977. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.286517 11347 tablet_bootstrap.cc:492] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:11.286877 11342 tablet_bootstrap.cc:654] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.287086 11344 tablet_bootstrap.cc:654] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.288162 11342 tablet_bootstrap.cc:492] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:11.288240 11342 ts_tablet_manager.cc:1367] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:11.288614 11342 raft_consensus.cc:348] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.288707 11342 raft_consensus.cc:374] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.288746 11342 raft_consensus.cc:729] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.288867 11342 consensus_queue.cc:260] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.289068 11347 tablet_bootstrap.cc:654] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.289165 11342 ts_tablet_manager.cc:1398] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:11.289319 11342 tablet_bootstrap.cc:492] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:11.290160 11342 tablet_bootstrap.cc:654] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.290242 11344 tablet_bootstrap.cc:492] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:11.290311 11347 tablet_bootstrap.cc:492] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:11.290323 11344 ts_tablet_manager.cc:1367] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:11.290386 11347 ts_tablet_manager.cc:1367] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 14:01:11.290756 11344 raft_consensus.cc:348] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.290833 11347 raft_consensus.cc:348] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.290872 11344 raft_consensus.cc:374] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.290938 11344 raft_consensus.cc:729] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.290946 11347 raft_consensus.cc:374] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.290987 11347 raft_consensus.cc:729] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.291061 11344 consensus_queue.cc:260] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.291115 11347 consensus_queue.cc:260] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.291345 11344 ts_tablet_manager.cc:1398] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:11.291393 11347 ts_tablet_manager.cc:1398] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:11.291492 11342 tablet_bootstrap.cc:492] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:11.291558 11342 ts_tablet_manager.cc:1367] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:11.291980 11342 raft_consensus.cc:348] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.292079 11342 raft_consensus.cc:374] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.292114 11344 tablet_bootstrap.cc:492] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:11.292117 11342 raft_consensus.cc:729] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.292227 11342 consensus_queue.cc:260] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.292461 11342 ts_tablet_manager.cc:1398] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:11.293087 11344 tablet_bootstrap.cc:654] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.294050 11343 tablet_bootstrap.cc:492] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:11.294149 11343 ts_tablet_manager.cc:1367] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 14:01:11.294327 11344 tablet_bootstrap.cc:492] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:11.294418 11344 ts_tablet_manager.cc:1367] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:11.294595 11343 raft_consensus.cc:348] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.294689 11343 raft_consensus.cc:374] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.294730 11343 raft_consensus.cc:729] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.294905 11343 consensus_queue.cc:260] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.295147 11343 ts_tablet_manager.cc:1398] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:11.295151 11344 raft_consensus.cc:348] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.295265 11344 raft_consensus.cc:374] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.295305 11344 raft_consensus.cc:729] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.295322 11343 tablet_bootstrap.cc:492] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:11.295445 11344 consensus_queue.cc:260] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.295809 11344 ts_tablet_manager.cc:1398] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:11.296216 11343 tablet_bootstrap.cc:654] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.297871 11343 tablet_bootstrap.cc:492] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:11.297966 11343 ts_tablet_manager.cc:1367] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:11.298362 11343 raft_consensus.cc:348] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.298447 11343 raft_consensus.cc:374] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.298481 11343 raft_consensus.cc:729] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.298609 11343 consensus_queue.cc:260] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.298861 11343 ts_tablet_manager.cc:1398] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:11.311887 11108 raft_consensus.cc:482] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:11.312011 11108 raft_consensus.cc:504] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.312335 11108 leader_election.cc:288] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:11.312726 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeceb15b637841609765e29e9b4d7ef1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:11.312772 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeceb15b637841609765e29e9b4d7ef1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:11.312857 10113 raft_consensus.cc:2454] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:11.312911 10507 raft_consensus.cc:2454] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:11.313141 10180 leader_election.cc:302] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:11.313321 11108 raft_consensus.cc:2781] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:11.313376 11108 raft_consensus.cc:482] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:11.313429 11108 raft_consensus.cc:3037] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:11.314363 11108 raft_consensus.cc:504] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.314666 11108 leader_election.cc:288] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:11.314994 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeceb15b637841609765e29e9b4d7ef1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:11.315032 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeceb15b637841609765e29e9b4d7ef1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:11.315093 10113 raft_consensus.cc:3037] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:11.315140 10507 raft_consensus.cc:3037] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:11.315881 10113 raft_consensus.cc:2454] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:11.315891 10507 raft_consensus.cc:2454] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:11.316169 10180 leader_election.cc:302] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:11.316334 11108 raft_consensus.cc:2781] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:11.316502 11108 raft_consensus.cc:686] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:11.316656 11108 consensus_queue.cc:237] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.318233 9971 catalog_manager.cc:5484] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 1 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:11.529434 11398 raft_consensus.cc:482] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:11.529570 11398 raft_consensus.cc:504] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.529966 11398 leader_election.cc:288] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:11.530375 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cf72901dd0434d9b88902343d45977" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:11.530407 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cf72901dd0434d9b88902343d45977" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:11.530536 10507 raft_consensus.cc:2454] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:11.530536 10113 raft_consensus.cc:2454] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:11.530845 10311 leader_election.cc:302] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:11.531016 11398 raft_consensus.cc:2781] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:11.531077 11398 raft_consensus.cc:482] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:11.531131 11398 raft_consensus.cc:3037] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:11.532219 11398 raft_consensus.cc:504] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.532565 11398 leader_election.cc:288] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:11.532929 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cf72901dd0434d9b88902343d45977" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:11.533038 10113 raft_consensus.cc:3037] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:11.533025 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cf72901dd0434d9b88902343d45977" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:11.533136 10507 raft_consensus.cc:3037] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:11.533779 10113 raft_consensus.cc:2454] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:11.533877 10507 raft_consensus.cc:2454] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:11.534044 10311 leader_election.cc:302] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:11.534188 11398 raft_consensus.cc:2781] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:11.534271 11398 raft_consensus.cc:686] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:11.534417 11398 consensus_queue.cc:237] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.536105 9971 catalog_manager.cc:5484] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:11.600709 11108 raft_consensus.cc:482] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:11.600837 11108 raft_consensus.cc:504] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.601208 11108 leader_election.cc:288] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:11.601682 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657170da057f4318a6165d2ed9097e58" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:11.601707 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657170da057f4318a6165d2ed9097e58" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:11.601832 10113 raft_consensus.cc:2454] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:11.601831 10376 raft_consensus.cc:2454] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:11.602149 10178 leader_election.cc:302] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:11.602320 11108 raft_consensus.cc:2781] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:11.602371 11108 raft_consensus.cc:482] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:11.602398 11108 raft_consensus.cc:3037] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:11.603199 11108 raft_consensus.cc:504] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.603533 11108 leader_election.cc:288] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:11.603868 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657170da057f4318a6165d2ed9097e58" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:11.603871 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657170da057f4318a6165d2ed9097e58" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:11.603967 10376 raft_consensus.cc:3037] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:11.603981 10113 raft_consensus.cc:3037] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:11.604725 10376 raft_consensus.cc:2454] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:11.604761 10113 raft_consensus.cc:2454] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:11.605010 10178 leader_election.cc:302] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:11.605165 11108 raft_consensus.cc:2781] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:11.605250 11108 raft_consensus.cc:686] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:11.605387 11108 consensus_queue.cc:237] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.607093 9971 catalog_manager.cc:5484] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 1 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:11.764253 11412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:11.764478 11412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240502 14:01:11.766201 11108 consensus_queue.cc:1035] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 14:01:11.767928 11412 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 14:01:11.770432 11108 consensus_queue.cc:1035] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_4'
I20240502 14:01:11.848625 10376 raft_consensus.cc:1261] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:11.849123 11408 consensus_queue.cc:1035] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:11.855381 10113 raft_consensus.cc:1261] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:11.855629 10508 raft_consensus.cc:1261] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:11.856401 10113 raft_consensus.cc:1261] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:11.856657 11247 consensus_queue.cc:1035] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:11.856932 11247 consensus_queue.cc:1035] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:11.857201 11108 consensus_queue.cc:1035] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 99.252 ms
time per row: 0.049626 ms
I20240502 14:01:11.913957 9964 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44294:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:01:11.919601 10356 tablet_service.cc:1467] Processing CreateTablet for tablet f1aa508f78bc40cdb8616017d9e0c5d8 (DEFAULT_TABLE table=rebalance_test_table_5 [id=36dba5bf6a854859adef03c1abe80f18]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.919740 10485 tablet_service.cc:1467] Processing CreateTablet for tablet f1aa508f78bc40cdb8616017d9e0c5d8 (DEFAULT_TABLE table=rebalance_test_table_5 [id=36dba5bf6a854859adef03c1abe80f18]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.919994 10356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1aa508f78bc40cdb8616017d9e0c5d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.920009 10485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1aa508f78bc40cdb8616017d9e0c5d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.920209 10224 tablet_service.cc:1467] Processing CreateTablet for tablet f1aa508f78bc40cdb8616017d9e0c5d8 (DEFAULT_TABLE table=rebalance_test_table_5 [id=36dba5bf6a854859adef03c1abe80f18]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.920444 10224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1aa508f78bc40cdb8616017d9e0c5d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.921583 10223 tablet_service.cc:1467] Processing CreateTablet for tablet 10bcbc135aa64d74bd5aaef9d4f2115d (DEFAULT_TABLE table=rebalance_test_table_5 [id=36dba5bf6a854859adef03c1abe80f18]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.921838 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10bcbc135aa64d74bd5aaef9d4f2115d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.922946 10092 tablet_service.cc:1467] Processing CreateTablet for tablet b0f4a67e31b04fefac38cf7d7cb983e7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=36dba5bf6a854859adef03c1abe80f18]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.923189 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0f4a67e31b04fefac38cf7d7cb983e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.923290 10225 tablet_service.cc:1467] Processing CreateTablet for tablet b0f4a67e31b04fefac38cf7d7cb983e7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=36dba5bf6a854859adef03c1abe80f18]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.923429 10485 tablet_service.cc:1467] Processing CreateTablet for tablet 10bcbc135aa64d74bd5aaef9d4f2115d (DEFAULT_TABLE table=rebalance_test_table_5 [id=36dba5bf6a854859adef03c1abe80f18]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.923496 10225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0f4a67e31b04fefac38cf7d7cb983e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.923652 10485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10bcbc135aa64d74bd5aaef9d4f2115d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.924361 10355 tablet_service.cc:1467] Processing CreateTablet for tablet 10bcbc135aa64d74bd5aaef9d4f2115d (DEFAULT_TABLE table=rebalance_test_table_5 [id=36dba5bf6a854859adef03c1abe80f18]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.924597 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10bcbc135aa64d74bd5aaef9d4f2115d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.925861 10354 tablet_service.cc:1467] Processing CreateTablet for tablet b0f4a67e31b04fefac38cf7d7cb983e7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=36dba5bf6a854859adef03c1abe80f18]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:11.926064 10354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0f4a67e31b04fefac38cf7d7cb983e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:11.927704 11472 tablet_bootstrap.cc:492] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:11.928689 11472 tablet_bootstrap.cc:654] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.930617 11471 tablet_bootstrap.cc:492] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:11.930882 11475 tablet_bootstrap.cc:492] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:11.931726 11473 tablet_bootstrap.cc:492] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:11.932090 11475 tablet_bootstrap.cc:654] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.932497 11472 tablet_bootstrap.cc:492] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:11.932600 11472 ts_tablet_manager.cc:1367] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:01:11.932614 11473 tablet_bootstrap.cc:654] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.933112 11472 raft_consensus.cc:348] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.933231 11472 raft_consensus.cc:374] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.933270 11472 raft_consensus.cc:729] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.933351 11471 tablet_bootstrap.cc:654] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.933375 11475 tablet_bootstrap.cc:492] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:11.933445 11475 ts_tablet_manager.cc:1367] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:11.933442 11472 consensus_queue.cc:260] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.933730 11472 ts_tablet_manager.cc:1398] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:11.933863 11475 raft_consensus.cc:348] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:11.933955 11475 raft_consensus.cc:374] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.933979 11475 raft_consensus.cc:729] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.934084 11475 consensus_queue.cc:260] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:11.934326 11471 tablet_bootstrap.cc:492] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:11.934345 11475 ts_tablet_manager.cc:1398] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:11.934409 11471 ts_tablet_manager.cc:1367] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:11.934494 11475 tablet_bootstrap.cc:492] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:11.934893 11471 raft_consensus.cc:348] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:11.935011 11471 raft_consensus.cc:374] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.935045 11471 raft_consensus.cc:729] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.935163 11471 consensus_queue.cc:260] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:11.935288 11475 tablet_bootstrap.cc:654] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.935448 11471 ts_tablet_manager.cc:1398] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:11.935607 11471 tablet_bootstrap.cc:492] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:11.936479 11471 tablet_bootstrap.cc:654] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.937963 11471 tablet_bootstrap.cc:492] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:11.938045 11471 ts_tablet_manager.cc:1367] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:11.938459 11471 raft_consensus.cc:348] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.938555 11471 raft_consensus.cc:374] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.938581 11471 raft_consensus.cc:729] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.938683 11471 consensus_queue.cc:260] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.938900 11471 ts_tablet_manager.cc:1398] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:11.940502 11475 tablet_bootstrap.cc:492] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:11.940587 11475 ts_tablet_manager.cc:1367] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:01:11.941004 11475 raft_consensus.cc:348] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.941109 11475 raft_consensus.cc:374] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.941130 11475 raft_consensus.cc:729] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.941254 11475 consensus_queue.cc:260] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.941479 11475 ts_tablet_manager.cc:1398] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:11.941646 11475 tablet_bootstrap.cc:492] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:11.942801 11475 tablet_bootstrap.cc:654] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.943759 11473 tablet_bootstrap.cc:492] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:11.943784 11475 tablet_bootstrap.cc:492] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:11.943893 11475 ts_tablet_manager.cc:1367] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:11.943903 11473 ts_tablet_manager.cc:1367] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240502 14:01:11.944347 11475 raft_consensus.cc:348] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.944372 11473 raft_consensus.cc:348] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:11.944461 11475 raft_consensus.cc:374] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.944482 11473 raft_consensus.cc:374] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.944492 11475 raft_consensus.cc:729] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.944514 11473 raft_consensus.cc:729] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.944581 11475 consensus_queue.cc:260] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.944656 11473 consensus_queue.cc:260] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:11.944845 11475 ts_tablet_manager.cc:1398] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:11.944957 11473 ts_tablet_manager.cc:1398] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:11.945168 11473 tablet_bootstrap.cc:492] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:11.945982 11473 tablet_bootstrap.cc:654] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.947223 11473 tablet_bootstrap.cc:492] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:11.947305 11473 ts_tablet_manager.cc:1367] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 14:01:11.947613 11473 raft_consensus.cc:348] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.947714 11473 raft_consensus.cc:374] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.947746 11473 raft_consensus.cc:729] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.947928 11473 consensus_queue.cc:260] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:11.948287 11473 ts_tablet_manager.cc:1398] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:11.948473 11473 tablet_bootstrap.cc:492] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:11.949193 11473 tablet_bootstrap.cc:654] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:11.950212 11473 tablet_bootstrap.cc:492] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:11.950291 11473 ts_tablet_manager.cc:1367] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:11.950642 11473 raft_consensus.cc:348] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.950747 11473 raft_consensus.cc:374] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:11.950778 11473 raft_consensus.cc:729] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:11.950886 11473 consensus_queue.cc:260] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:11.951098 11473 ts_tablet_manager.cc:1398] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:12.036132 11460 raft_consensus.cc:482] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:12.036290 11460 raft_consensus.cc:504] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:12.036782 11460 leader_election.cc:288] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:12.037281 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f4a67e31b04fefac38cf7d7cb983e7" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:12.037281 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f4a67e31b04fefac38cf7d7cb983e7" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:12.037431 10376 raft_consensus.cc:2454] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:12.037448 10245 raft_consensus.cc:2454] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:12.037781 10047 leader_election.cc:302] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:12.037948 11460 raft_consensus.cc:2781] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:12.038004 11460 raft_consensus.cc:482] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:12.038030 11460 raft_consensus.cc:3037] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.038913 11460 raft_consensus.cc:504] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:12.039219 11460 leader_election.cc:288] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:12.039573 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f4a67e31b04fefac38cf7d7cb983e7" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:12.039616 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f4a67e31b04fefac38cf7d7cb983e7" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:12.039683 10245 raft_consensus.cc:3037] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.039762 10376 raft_consensus.cc:3037] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.040643 10245 raft_consensus.cc:2454] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:12.040664 10376 raft_consensus.cc:2454] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:12.041000 10047 leader_election.cc:302] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:12.041203 11460 raft_consensus.cc:2781] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:12.041293 11460 raft_consensus.cc:686] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Leader. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:12.041442 11460 consensus_queue.cc:237] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:12.043177 9964 catalog_manager.cc:5484] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be reported cstate change: term changed from 0 to 1, leader changed from <none> to 068f58913ea3478599df829d5118f7be (127.26.25.129). New cstate: current_term: 1 leader_uuid: "068f58913ea3478599df829d5118f7be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:12.124313 11247 raft_consensus.cc:482] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:12.124459 11247 raft_consensus.cc:504] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.124897 11247 leader_election.cc:288] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:12.125397 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:12.125432 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:12.125543 10507 raft_consensus.cc:2454] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:12.125555 10245 raft_consensus.cc:2454] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:12.125854 10311 leader_election.cc:302] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:12.126037 11247 raft_consensus.cc:2781] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:12.126091 11247 raft_consensus.cc:482] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:12.126119 11247 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.126935 11247 raft_consensus.cc:504] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.127245 11247 leader_election.cc:288] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:12.127595 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:12.127616 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:12.127701 10507 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.127712 10245 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.128472 10245 raft_consensus.cc:2454] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:12.128490 10507 raft_consensus.cc:2454] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:12.128778 10311 leader_election.cc:302] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:12.128945 11247 raft_consensus.cc:2781] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:12.129024 11247 raft_consensus.cc:686] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:12.129168 11247 consensus_queue.cc:237] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.130903 9964 catalog_manager.cc:5484] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:12.218755 11247 raft_consensus.cc:482] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:12.218894 11247 raft_consensus.cc:504] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.219319 11247 leader_election.cc:288] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:12.219759 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:12.219856 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:12.219938 10245 raft_consensus.cc:2454] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:12.220022 10507 raft_consensus.cc:2454] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:12.220278 10311 leader_election.cc:302] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:12.220471 11247 raft_consensus.cc:2781] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:12.220533 11247 raft_consensus.cc:482] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:12.220561 11247 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.221377 11247 raft_consensus.cc:504] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.221693 11247 leader_election.cc:288] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:12.222077 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:12.222072 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:12.222185 10507 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.222191 10245 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.222940 10245 raft_consensus.cc:2454] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:12.222941 10507 raft_consensus.cc:2454] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:12.223222 10311 leader_election.cc:302] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:12.223378 11247 raft_consensus.cc:2781] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:12.223512 11247 raft_consensus.cc:686] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:12.223647 11247 consensus_queue.cc:237] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.225454 9964 catalog_manager.cc:5484] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:12.407049 11487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:12.407279 11487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:12.410352 11487 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 14:01:12.486086 10507 raft_consensus.cc:1261] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:12.486500 10245 raft_consensus.cc:1261] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:12.487072 11247 consensus_queue.cc:1035] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:12.487530 10376 raft_consensus.cc:1261] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:12.487962 11247 consensus_queue.cc:1035] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:12.487960 11460 consensus_queue.cc:1035] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:12.489523 10244 raft_consensus.cc:1261] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:12.490864 11398 consensus_queue.cc:1035] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:12.491539 10244 raft_consensus.cc:1261] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:12.492249 11460 consensus_queue.cc:1035] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:12.492383 10508 raft_consensus.cc:1261] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:12.493759 11398 consensus_queue.cc:1035] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:12.505661 11518 mvcc.cc:205] Tried to move back new op lower bound from 7023241103304638464 to 7023241101482844160. Current Snapshot: MvccSnapshot[applied={T|T < 7023241103304638464}]
I20240502 14:01:12.522045 11528 mvcc.cc:205] Tried to move back new op lower bound from 7023241103304638464 to 7023241101482844160. Current Snapshot: MvccSnapshot[applied={T|T < 7023241103304638464}]
I20240502 14:01:12.532651 11531 mvcc.cc:205] Tried to move back new op lower bound from 7023241103325556736 to 7023241101842305024. Current Snapshot: MvccSnapshot[applied={T|T < 7023241103311695872}]
INSERT report
rows total: 2000
time total: 93.4693 ms
time per row: 0.0467347 ms
I20240502 14:01:12.548903 9971 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44294:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:01:12.554483 10485 tablet_service.cc:1467] Processing CreateTablet for tablet a54336e39ef249fdb7eb6b50cb5b1cda (DEFAULT_TABLE table=rebalance_test_table_6 [id=015f5df23d6e400ab6f376ef603dcd2c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:12.554627 10092 tablet_service.cc:1467] Processing CreateTablet for tablet a54336e39ef249fdb7eb6b50cb5b1cda (DEFAULT_TABLE table=rebalance_test_table_6 [id=015f5df23d6e400ab6f376ef603dcd2c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:12.554828 10486 tablet_service.cc:1467] Processing CreateTablet for tablet 0a820d81837c49c3a6a99b09dd5afc49 (DEFAULT_TABLE table=rebalance_test_table_6 [id=015f5df23d6e400ab6f376ef603dcd2c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:12.554843 10485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54336e39ef249fdb7eb6b50cb5b1cda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:12.554946 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54336e39ef249fdb7eb6b50cb5b1cda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:12.555091 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a820d81837c49c3a6a99b09dd5afc49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:12.555606 10223 tablet_service.cc:1467] Processing CreateTablet for tablet a54336e39ef249fdb7eb6b50cb5b1cda (DEFAULT_TABLE table=rebalance_test_table_6 [id=015f5df23d6e400ab6f376ef603dcd2c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:12.555895 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54336e39ef249fdb7eb6b50cb5b1cda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:12.556103 10487 tablet_service.cc:1467] Processing CreateTablet for tablet 965596e659e340c7b231e4eef1868e92 (DEFAULT_TABLE table=rebalance_test_table_6 [id=015f5df23d6e400ab6f376ef603dcd2c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:12.556334 10487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 965596e659e340c7b231e4eef1868e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:12.557013 10355 tablet_service.cc:1467] Processing CreateTablet for tablet 0a820d81837c49c3a6a99b09dd5afc49 (DEFAULT_TABLE table=rebalance_test_table_6 [id=015f5df23d6e400ab6f376ef603dcd2c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:12.557128 10225 tablet_service.cc:1467] Processing CreateTablet for tablet 965596e659e340c7b231e4eef1868e92 (DEFAULT_TABLE table=rebalance_test_table_6 [id=015f5df23d6e400ab6f376ef603dcd2c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:12.557271 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a820d81837c49c3a6a99b09dd5afc49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:12.557274 11546 tablet_bootstrap.cc:492] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:12.557361 10225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 965596e659e340c7b231e4eef1868e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:12.558064 11547 tablet_bootstrap.cc:492] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:12.558238 11546 tablet_bootstrap.cc:654] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:12.558919 11547 tablet_bootstrap.cc:654] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:12.559597 11549 tablet_bootstrap.cc:492] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:12.559710 10092 tablet_service.cc:1467] Processing CreateTablet for tablet 0a820d81837c49c3a6a99b09dd5afc49 (DEFAULT_TABLE table=rebalance_test_table_6 [id=015f5df23d6e400ab6f376ef603dcd2c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:12.559775 10094 tablet_service.cc:1467] Processing CreateTablet for tablet 965596e659e340c7b231e4eef1868e92 (DEFAULT_TABLE table=rebalance_test_table_6 [id=015f5df23d6e400ab6f376ef603dcd2c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:12.559970 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a820d81837c49c3a6a99b09dd5afc49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:12.560143 10094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 965596e659e340c7b231e4eef1868e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:12.560489 11549 tablet_bootstrap.cc:654] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:12.561664 11549 tablet_bootstrap.cc:492] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:12.561764 11549 ts_tablet_manager.cc:1367] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:12.562247 11549 raft_consensus.cc:348] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.562381 11549 raft_consensus.cc:374] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:12.562453 11549 raft_consensus.cc:729] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:12.562620 11549 consensus_queue.cc:260] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.563150 11546 tablet_bootstrap.cc:492] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:12.563242 11546 ts_tablet_manager.cc:1367] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:01:12.563699 11546 raft_consensus.cc:348] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.563792 11546 raft_consensus.cc:374] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:12.563855 11546 raft_consensus.cc:729] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:12.563953 11547 tablet_bootstrap.cc:492] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:12.563994 11546 consensus_queue.cc:260] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.564046 11547 ts_tablet_manager.cc:1367] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:01:12.564249 11546 ts_tablet_manager.cc:1398] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:12.564378 11546 tablet_bootstrap.cc:492] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:12.565124 11547 raft_consensus.cc:348] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.565217 11546 tablet_bootstrap.cc:654] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:12.565225 11547 raft_consensus.cc:374] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:12.565266 11547 raft_consensus.cc:729] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:12.565400 11547 consensus_queue.cc:260] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.565691 11547 ts_tablet_manager.cc:1398] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 14:01:12.565867 11547 tablet_bootstrap.cc:492] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:12.566220 11549 ts_tablet_manager.cc:1398] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:01:12.566913 11546 tablet_bootstrap.cc:492] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:12.566991 11546 ts_tablet_manager.cc:1367] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:12.567209 11547 tablet_bootstrap.cc:654] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:12.567412 11546 raft_consensus.cc:348] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.567512 11546 raft_consensus.cc:374] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:12.567548 11546 raft_consensus.cc:729] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:12.567659 11546 consensus_queue.cc:260] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.567987 11546 ts_tablet_manager.cc:1398] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:12.568132 11546 tablet_bootstrap.cc:492] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:12.568290 11547 tablet_bootstrap.cc:492] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:12.568378 11547 ts_tablet_manager.cc:1367] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:12.568802 11547 raft_consensus.cc:348] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.568897 11547 raft_consensus.cc:374] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:12.568930 11547 raft_consensus.cc:729] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:12.568981 11546 tablet_bootstrap.cc:654] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:12.569082 11547 consensus_queue.cc:260] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.569406 11547 ts_tablet_manager.cc:1398] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:12.569976 11546 tablet_bootstrap.cc:492] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:12.570055 11546 ts_tablet_manager.cc:1367] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:12.570433 11546 raft_consensus.cc:348] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.570515 11546 raft_consensus.cc:374] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:12.570549 11546 raft_consensus.cc:729] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:12.570695 11546 consensus_queue.cc:260] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.570947 11546 ts_tablet_manager.cc:1398] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:12.574128 11555 tablet_bootstrap.cc:492] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:12.574891 11555 tablet_bootstrap.cc:654] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:12.575896 11555 tablet_bootstrap.cc:492] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:12.575987 11555 ts_tablet_manager.cc:1367] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:12.576478 11555 raft_consensus.cc:348] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.576573 11555 raft_consensus.cc:374] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:12.576608 11555 raft_consensus.cc:729] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:12.576756 11555 consensus_queue.cc:260] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.577064 11555 ts_tablet_manager.cc:1398] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:12.577276 11555 tablet_bootstrap.cc:492] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:12.578224 11555 tablet_bootstrap.cc:654] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:12.579306 11555 tablet_bootstrap.cc:492] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:12.579392 11555 ts_tablet_manager.cc:1367] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:12.579856 11555 raft_consensus.cc:348] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.579954 11555 raft_consensus.cc:374] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:12.579988 11555 raft_consensus.cc:729] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:12.580112 11555 consensus_queue.cc:260] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.580380 11555 ts_tablet_manager.cc:1398] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:12.580544 11555 tablet_bootstrap.cc:492] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:12.581682 11555 tablet_bootstrap.cc:654] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:12.582791 11555 tablet_bootstrap.cc:492] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:12.582881 11555 ts_tablet_manager.cc:1367] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:12.583292 11555 raft_consensus.cc:348] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.583389 11555 raft_consensus.cc:374] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:12.583422 11555 raft_consensus.cc:729] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:12.583556 11555 consensus_queue.cc:260] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.583856 11555 ts_tablet_manager.cc:1398] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:12.689509 11398 raft_consensus.cc:482] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:12.689657 11398 raft_consensus.cc:504] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.690130 11398 leader_election.cc:288] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:12.690652 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a820d81837c49c3a6a99b09dd5afc49" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:12.690658 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a820d81837c49c3a6a99b09dd5afc49" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:12.690811 10507 raft_consensus.cc:2454] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:12.690811 10112 raft_consensus.cc:2454] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:12.691144 10311 leader_election.cc:302] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:12.691345 11247 raft_consensus.cc:2781] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:12.691408 11247 raft_consensus.cc:482] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:12.691449 11247 raft_consensus.cc:3037] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.692325 11247 raft_consensus.cc:504] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.692629 11247 leader_election.cc:288] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:12.693038 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a820d81837c49c3a6a99b09dd5afc49" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:12.693094 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a820d81837c49c3a6a99b09dd5afc49" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:12.693161 10507 raft_consensus.cc:3037] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.693250 10112 raft_consensus.cc:3037] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.693899 10507 raft_consensus.cc:2454] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:12.694078 10112 raft_consensus.cc:2454] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:12.694192 10311 leader_election.cc:302] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:12.694360 11247 raft_consensus.cc:2781] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:12.694439 11247 raft_consensus.cc:686] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:12.694583 11247 consensus_queue.cc:237] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:12.696413 9964 catalog_manager.cc:5484] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:12.787106 11125 raft_consensus.cc:482] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:12.787240 11125 raft_consensus.cc:504] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.787660 11125 leader_election.cc:288] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:12.788070 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54336e39ef249fdb7eb6b50cb5b1cda" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:12.788223 10507 raft_consensus.cc:2454] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:12.788177 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54336e39ef249fdb7eb6b50cb5b1cda" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:12.788352 10245 raft_consensus.cc:2454] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:12.788533 10049 leader_election.cc:302] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:12.788705 11125 raft_consensus.cc:2781] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:12.788766 11125 raft_consensus.cc:482] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:12.788813 11125 raft_consensus.cc:3037] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.790052 11125 raft_consensus.cc:504] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.790393 11125 leader_election.cc:288] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:12.790690 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54336e39ef249fdb7eb6b50cb5b1cda" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:12.790789 10507 raft_consensus.cc:3037] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.790782 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54336e39ef249fdb7eb6b50cb5b1cda" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:12.790906 10245 raft_consensus.cc:3037] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.791616 10507 raft_consensus.cc:2454] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:12.791646 10245 raft_consensus.cc:2454] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:12.791905 10049 leader_election.cc:302] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:12.792089 11125 raft_consensus.cc:2781] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:12.792171 11125 raft_consensus.cc:686] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Leader. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:12.792349 11125 consensus_queue.cc:237] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.794144 9964 catalog_manager.cc:5484] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be reported cstate change: term changed from 0 to 1, leader changed from <none> to 068f58913ea3478599df829d5118f7be (127.26.25.129). New cstate: current_term: 1 leader_uuid: "068f58913ea3478599df829d5118f7be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:12.810943 11125 raft_consensus.cc:482] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:12.811058 11125 raft_consensus.cc:504] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.811424 11125 leader_election.cc:288] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:12.811882 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965596e659e340c7b231e4eef1868e92" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:12.811939 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965596e659e340c7b231e4eef1868e92" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:12.812026 10245 raft_consensus.cc:2454] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:12.812096 10507 raft_consensus.cc:2454] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:12.812299 10049 leader_election.cc:302] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:12.812477 11125 raft_consensus.cc:2781] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:12.812536 11125 raft_consensus.cc:482] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:12.812592 11125 raft_consensus.cc:3037] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.813460 11125 raft_consensus.cc:504] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.813783 11125 leader_election.cc:288] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:12.814146 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965596e659e340c7b231e4eef1868e92" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:12.814204 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965596e659e340c7b231e4eef1868e92" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:12.814256 10507 raft_consensus.cc:3037] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.814322 10245 raft_consensus.cc:3037] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:12.815029 10507 raft_consensus.cc:2454] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:12.815075 10245 raft_consensus.cc:2454] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:12.815318 10049 leader_election.cc:302] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:12.815480 11125 raft_consensus.cc:2781] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:12.815567 11125 raft_consensus.cc:686] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Leader. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:12.815704 11125 consensus_queue.cc:237] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:12.817608 9964 catalog_manager.cc:5484] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be reported cstate change: term changed from 0 to 1, leader changed from <none> to 068f58913ea3478599df829d5118f7be (127.26.25.129). New cstate: current_term: 1 leader_uuid: "068f58913ea3478599df829d5118f7be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:12.964447 11562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:12.964744 11562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:12.967856 11562 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 14:01:13.045110 10507 raft_consensus.cc:1261] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:13.045861 11247 consensus_queue.cc:1035] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:13.047103 10245 raft_consensus.cc:1261] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:13.047524 10508 raft_consensus.cc:1261] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:13.047932 10508 raft_consensus.cc:1261] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:13.048029 10245 raft_consensus.cc:1261] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:13.048800 11460 consensus_queue.cc:1035] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:13.048938 10112 raft_consensus.cc:1261] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:13.049093 11460 consensus_queue.cc:1035] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:13.049326 11460 consensus_queue.cc:1035] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:13.049587 11460 consensus_queue.cc:1035] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:13.051079 11247 consensus_queue.cc:1035] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:13.064188 11591 mvcc.cc:205] Tried to move back new op lower bound from 7023241105588817920 to 7023241104158400512. Current Snapshot: MvccSnapshot[applied={T|T < 7023241105588817920}]
INSERT report
rows total: 2000
time total: 107.674 ms
time per row: 0.0538369 ms
I20240502 14:01:13.120715 9964 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44294:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:01:13.126267 10092 tablet_service.cc:1467] Processing CreateTablet for tablet fb32f5457e4345a68b283be8891ce202 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ab44accb006481088b49b18cc5363af]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.126237 10355 tablet_service.cc:1467] Processing CreateTablet for tablet fb32f5457e4345a68b283be8891ce202 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ab44accb006481088b49b18cc5363af]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.126410 10225 tablet_service.cc:1467] Processing CreateTablet for tablet fb32f5457e4345a68b283be8891ce202 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ab44accb006481088b49b18cc5363af]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.126567 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb32f5457e4345a68b283be8891ce202. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.126567 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb32f5457e4345a68b283be8891ce202. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.126700 10225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb32f5457e4345a68b283be8891ce202. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.128909 11624 tablet_bootstrap.cc:492] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:13.129190 10486 tablet_service.cc:1467] Processing CreateTablet for tablet 2379dbd40b744991be25c2914f878913 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ab44accb006481088b49b18cc5363af]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.129190 10487 tablet_service.cc:1467] Processing CreateTablet for tablet bb4efdb1ee0d43558cdc3106829b159c (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ab44accb006481088b49b18cc5363af]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.129480 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2379dbd40b744991be25c2914f878913. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.129758 10225 tablet_service.cc:1467] Processing CreateTablet for tablet bb4efdb1ee0d43558cdc3106829b159c (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ab44accb006481088b49b18cc5363af]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.129839 11624 tablet_bootstrap.cc:654] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.129971 10225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb4efdb1ee0d43558cdc3106829b159c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.130445 11625 tablet_bootstrap.cc:492] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:13.130380 10354 tablet_service.cc:1467] Processing CreateTablet for tablet bb4efdb1ee0d43558cdc3106829b159c (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ab44accb006481088b49b18cc5363af]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.130628 10354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb4efdb1ee0d43558cdc3106829b159c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.130694 10487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb4efdb1ee0d43558cdc3106829b159c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.131279 11625 tablet_bootstrap.cc:654] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.132189 10355 tablet_service.cc:1467] Processing CreateTablet for tablet 2379dbd40b744991be25c2914f878913 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ab44accb006481088b49b18cc5363af]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.132288 11625 tablet_bootstrap.cc:492] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:13.132364 11625 ts_tablet_manager.cc:1367] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:13.132426 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2379dbd40b744991be25c2914f878913. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.132762 11625 raft_consensus.cc:348] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.132858 11625 raft_consensus.cc:374] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.132885 11625 raft_consensus.cc:729] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.133023 11625 consensus_queue.cc:260] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.133330 11625 ts_tablet_manager.cc:1398] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.133462 11627 tablet_bootstrap.cc:492] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:13.134328 11627 tablet_bootstrap.cc:654] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.134447 11625 tablet_bootstrap.cc:492] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:13.135365 11625 tablet_bootstrap.cc:654] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.135557 10092 tablet_service.cc:1467] Processing CreateTablet for tablet 2379dbd40b744991be25c2914f878913 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ab44accb006481088b49b18cc5363af]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.135766 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2379dbd40b744991be25c2914f878913. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.136656 11625 tablet_bootstrap.cc:492] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:13.136739 11625 ts_tablet_manager.cc:1367] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:13.137130 11625 raft_consensus.cc:348] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.137221 11625 raft_consensus.cc:374] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.137257 11625 raft_consensus.cc:729] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.137406 11625 consensus_queue.cc:260] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.137707 11623 tablet_bootstrap.cc:492] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:13.138152 11625 ts_tablet_manager.cc:1398] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.138365 11625 tablet_bootstrap.cc:492] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:13.138618 11623 tablet_bootstrap.cc:654] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.139864 11625 tablet_bootstrap.cc:654] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.139881 11624 tablet_bootstrap.cc:492] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:13.139940 11623 tablet_bootstrap.cc:492] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:13.139968 11624 ts_tablet_manager.cc:1367] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:01:13.140028 11627 tablet_bootstrap.cc:492] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:13.140058 11623 ts_tablet_manager.cc:1367] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:13.140108 11627 ts_tablet_manager.cc:1367] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:01:13.140471 11624 raft_consensus.cc:348] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.140509 11627 raft_consensus.cc:348] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.140578 11624 raft_consensus.cc:374] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.140599 11627 raft_consensus.cc:374] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.140611 11624 raft_consensus.cc:729] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.140632 11627 raft_consensus.cc:729] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.140743 11627 consensus_queue.cc:260] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.140739 11624 consensus_queue.cc:260] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.141010 11627 ts_tablet_manager.cc:1398] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:13.141010 11624 ts_tablet_manager.cc:1398] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.141016 11625 tablet_bootstrap.cc:492] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:13.141093 11625 ts_tablet_manager.cc:1367] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:13.141167 11624 tablet_bootstrap.cc:492] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:13.141167 11627 tablet_bootstrap.cc:492] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:13.141482 11625 raft_consensus.cc:348] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.141583 11625 raft_consensus.cc:374] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.141621 11625 raft_consensus.cc:729] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.140623 11623 raft_consensus.cc:348] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.141714 11623 raft_consensus.cc:374] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.141750 11623 raft_consensus.cc:729] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.141736 11625 consensus_queue.cc:260] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.141865 11623 consensus_queue.cc:260] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.141999 11624 tablet_bootstrap.cc:654] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.141999 11627 tablet_bootstrap.cc:654] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.142012 11625 ts_tablet_manager.cc:1398] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.142539 11623 ts_tablet_manager.cc:1398] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:13.142691 11623 tablet_bootstrap.cc:492] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:13.143508 11624 tablet_bootstrap.cc:492] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:13.143594 11627 tablet_bootstrap.cc:492] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:13.143678 11627 ts_tablet_manager.cc:1367] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:13.144155 11627 raft_consensus.cc:348] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.144258 11627 raft_consensus.cc:374] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.144296 11627 raft_consensus.cc:729] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.144404 11627 consensus_queue.cc:260] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.144837 11624 ts_tablet_manager.cc:1367] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:13.144968 11627 ts_tablet_manager.cc:1398] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.145076 11623 tablet_bootstrap.cc:654] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.145429 11624 raft_consensus.cc:348] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.145602 11624 raft_consensus.cc:374] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.145691 11624 raft_consensus.cc:729] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.145815 11624 consensus_queue.cc:260] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.146111 11624 ts_tablet_manager.cc:1398] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.149559 11623 tablet_bootstrap.cc:492] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:13.149637 11623 ts_tablet_manager.cc:1367] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:01:13.150066 11623 raft_consensus.cc:348] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.150159 11623 raft_consensus.cc:374] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.150180 11623 raft_consensus.cc:729] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.150312 11623 consensus_queue.cc:260] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.150591 11623 ts_tablet_manager.cc:1398] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:13.168880 11615 raft_consensus.cc:482] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:13.169047 11615 raft_consensus.cc:504] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.169507 11615 leader_election.cc:288] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:13.169826 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:13.169852 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:13.169971 10506 raft_consensus.cc:2454] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:13.169971 10376 raft_consensus.cc:2454] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:13.170260 10180 leader_election.cc:302] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:13.170467 11615 raft_consensus.cc:2781] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:13.170526 11615 raft_consensus.cc:482] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:13.170558 11615 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.171422 11615 raft_consensus.cc:504] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.171741 11615 leader_election.cc:288] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:13.172101 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:13.172101 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:13.172221 10376 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.172221 10506 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.173172 10376 raft_consensus.cc:2454] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:13.173172 10506 raft_consensus.cc:2454] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:13.173467 10178 leader_election.cc:302] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:13.173632 11615 raft_consensus.cc:2781] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:13.173719 11615 raft_consensus.cc:686] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:13.173861 11615 consensus_queue.cc:237] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.175597 9969 catalog_manager.cc:5484] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 1 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:13.273216 11597 raft_consensus.cc:482] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:13.273380 11597 raft_consensus.cc:504] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.273923 11597 leader_election.cc:288] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:13.274437 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2379dbd40b744991be25c2914f878913" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:13.274434 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2379dbd40b744991be25c2914f878913" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:13.274619 10376 raft_consensus.cc:2454] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 0.
I20240502 14:01:13.274619 10112 raft_consensus.cc:2454] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 0.
I20240502 14:01:13.274974 10442 leader_election.cc:302] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:13.275195 11597 raft_consensus.cc:2781] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:13.275269 11597 raft_consensus.cc:482] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:13.275307 11597 raft_consensus.cc:3037] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.276253 11597 raft_consensus.cc:504] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.276594 11597 leader_election.cc:288] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:13.276937 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2379dbd40b744991be25c2914f878913" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:13.277035 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2379dbd40b744991be25c2914f878913" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:13.277073 10376 raft_consensus.cc:3037] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.277160 10112 raft_consensus.cc:3037] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.278065 10376 raft_consensus.cc:2454] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 1.
I20240502 14:01:13.278065 10112 raft_consensus.cc:2454] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 1.
I20240502 14:01:13.278368 10442 leader_election.cc:302] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:13.278568 11597 raft_consensus.cc:2781] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:13.278674 11597 raft_consensus.cc:686] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:13.279027 11597 consensus_queue.cc:237] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.280927 9970 catalog_manager.cc:5484] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:13.436153 11595 raft_consensus.cc:482] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:13.436291 11595 raft_consensus.cc:504] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.436712 11595 leader_election.cc:288] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:13.437005 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb32f5457e4345a68b283be8891ce202" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:13.437088 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb32f5457e4345a68b283be8891ce202" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:13.437144 10376 raft_consensus.cc:2454] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:13.437248 10243 raft_consensus.cc:2454] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:13.437436 10047 leader_election.cc:302] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:13.437629 11595 raft_consensus.cc:2781] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:13.437686 11595 raft_consensus.cc:482] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:13.437711 11595 raft_consensus.cc:3037] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.438606 11595 raft_consensus.cc:504] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.438951 11595 leader_election.cc:288] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:13.439280 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb32f5457e4345a68b283be8891ce202" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:13.439384 10376 raft_consensus.cc:3037] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.439368 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb32f5457e4345a68b283be8891ce202" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:13.439476 10243 raft_consensus.cc:3037] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.440284 10376 raft_consensus.cc:2454] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:13.440351 10243 raft_consensus.cc:2454] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:13.440570 10047 leader_election.cc:302] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:13.440761 11595 raft_consensus.cc:2781] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:13.440852 11595 raft_consensus.cc:686] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Leader. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:13.440994 11595 consensus_queue.cc:237] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.442749 9970 catalog_manager.cc:5484] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be reported cstate change: term changed from 0 to 1, leader changed from <none> to 068f58913ea3478599df829d5118f7be (127.26.25.129). New cstate: current_term: 1 leader_uuid: "068f58913ea3478599df829d5118f7be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:13.611219 11639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:13.611519 11639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:13.614771 11639 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240502 14:01:13.655577 11615 consensus_queue.cc:1035] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:13.663239 11448 consensus_queue.cc:1035] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:13.703274 10376 raft_consensus.cc:1261] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:13.703516 10112 raft_consensus.cc:1261] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:13.704032 10243 raft_consensus.cc:1261] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:13.704125 10376 raft_consensus.cc:1261] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:13.704469 11595 consensus_queue.cc:1035] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:13.704566 11597 consensus_queue.cc:1035] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:13.704841 11597 consensus_queue.cc:1035] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:13.711118 11595 consensus_queue.cc:1035] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 113.587 ms
time per row: 0.0567934 ms
I20240502 14:01:13.774677 9972 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44294:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:01:13.780652 10092 tablet_service.cc:1467] Processing CreateTablet for tablet 0577d6cbf5a04722843d7ecf9cf28be9 (DEFAULT_TABLE table=rebalance_test_table_8 [id=7cc38cfd687349b39f70f17bd6863c66]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.780652 10355 tablet_service.cc:1467] Processing CreateTablet for tablet 0577d6cbf5a04722843d7ecf9cf28be9 (DEFAULT_TABLE table=rebalance_test_table_8 [id=7cc38cfd687349b39f70f17bd6863c66]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.780967 10092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0577d6cbf5a04722843d7ecf9cf28be9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.780967 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0577d6cbf5a04722843d7ecf9cf28be9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.782395 10094 tablet_service.cc:1467] Processing CreateTablet for tablet 9c1b9c544e23474791c93f4094b4c672 (DEFAULT_TABLE table=rebalance_test_table_8 [id=7cc38cfd687349b39f70f17bd6863c66]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.782681 10094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c1b9c544e23474791c93f4094b4c672. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.782851 10223 tablet_service.cc:1467] Processing CreateTablet for tablet 9c1b9c544e23474791c93f4094b4c672 (DEFAULT_TABLE table=rebalance_test_table_8 [id=7cc38cfd687349b39f70f17bd6863c66]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.783126 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c1b9c544e23474791c93f4094b4c672. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.783414 10225 tablet_service.cc:1467] Processing CreateTablet for tablet 0577d6cbf5a04722843d7ecf9cf28be9 (DEFAULT_TABLE table=rebalance_test_table_8 [id=7cc38cfd687349b39f70f17bd6863c66]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.783416 10487 tablet_service.cc:1467] Processing CreateTablet for tablet 9c1b9c544e23474791c93f4094b4c672 (DEFAULT_TABLE table=rebalance_test_table_8 [id=7cc38cfd687349b39f70f17bd6863c66]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.783634 10225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0577d6cbf5a04722843d7ecf9cf28be9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.783641 10487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c1b9c544e23474791c93f4094b4c672. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.784057 10486 tablet_service.cc:1467] Processing CreateTablet for tablet c958a9027a064dcb9026608541a22883 (DEFAULT_TABLE table=rebalance_test_table_8 [id=7cc38cfd687349b39f70f17bd6863c66]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.784255 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c958a9027a064dcb9026608541a22883. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.785274 11705 tablet_bootstrap.cc:492] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:13.782395 10093 tablet_service.cc:1467] Processing CreateTablet for tablet c958a9027a064dcb9026608541a22883 (DEFAULT_TABLE table=rebalance_test_table_8 [id=7cc38cfd687349b39f70f17bd6863c66]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.785547 10093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c958a9027a064dcb9026608541a22883. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.786266 11705 tablet_bootstrap.cc:654] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.787710 11706 tablet_bootstrap.cc:492] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:13.788398 11705 tablet_bootstrap.cc:492] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:13.788506 11705 ts_tablet_manager.cc:1367] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:13.789024 11706 tablet_bootstrap.cc:654] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.789175 11705 raft_consensus.cc:348] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.789217 10355 tablet_service.cc:1467] Processing CreateTablet for tablet c958a9027a064dcb9026608541a22883 (DEFAULT_TABLE table=rebalance_test_table_8 [id=7cc38cfd687349b39f70f17bd6863c66]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.789297 11705 raft_consensus.cc:374] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.789342 11705 raft_consensus.cc:729] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.789469 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c958a9027a064dcb9026608541a22883. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.789512 11705 consensus_queue.cc:260] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.789824 11705 ts_tablet_manager.cc:1398] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.789884 11704 tablet_bootstrap.cc:492] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:13.790004 11705 tablet_bootstrap.cc:492] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:13.790848 11705 tablet_bootstrap.cc:654] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.791795 11704 tablet_bootstrap.cc:654] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.792033 11703 tablet_bootstrap.cc:492] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:13.792191 11705 tablet_bootstrap.cc:492] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:13.792259 11705 ts_tablet_manager.cc:1367] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:13.792832 11705 raft_consensus.cc:348] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.792958 11705 raft_consensus.cc:374] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.792976 11703 tablet_bootstrap.cc:654] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.792990 11705 raft_consensus.cc:729] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.793144 11705 consensus_queue.cc:260] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.793519 11705 ts_tablet_manager.cc:1398] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.793603 11704 tablet_bootstrap.cc:492] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:13.793677 11704 ts_tablet_manager.cc:1367] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:13.794154 11704 raft_consensus.cc:348] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.794247 11704 raft_consensus.cc:374] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.794279 11704 raft_consensus.cc:729] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.794425 11706 tablet_bootstrap.cc:492] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:13.794425 11703 tablet_bootstrap.cc:492] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:13.794418 11704 consensus_queue.cc:260] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.794510 11706 ts_tablet_manager.cc:1367] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:01:13.794517 11703 ts_tablet_manager.cc:1367] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:13.794981 11706 raft_consensus.cc:348] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.795109 11706 raft_consensus.cc:374] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.795143 11706 raft_consensus.cc:729] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.795258 11706 consensus_queue.cc:260] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.795583 11704 ts_tablet_manager.cc:1398] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:13.795725 11704 tablet_bootstrap.cc:492] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:13.795761 11703 raft_consensus.cc:348] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.795913 11703 raft_consensus.cc:374] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.795956 11703 raft_consensus.cc:729] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.796072 11703 consensus_queue.cc:260] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.796118 11706 ts_tablet_manager.cc:1398] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:13.796288 11706 tablet_bootstrap.cc:492] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:13.796428 11703 ts_tablet_manager.cc:1398] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:13.796583 11704 tablet_bootstrap.cc:654] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.796620 11703 tablet_bootstrap.cc:492] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:13.797549 11706 tablet_bootstrap.cc:654] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.797935 11704 tablet_bootstrap.cc:492] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:13.798013 11704 ts_tablet_manager.cc:1367] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:13.798241 11703 tablet_bootstrap.cc:654] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.798513 11704 raft_consensus.cc:348] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.798614 11704 raft_consensus.cc:374] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.798630 11706 tablet_bootstrap.cc:492] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:13.798650 11704 raft_consensus.cc:729] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.798700 11706 ts_tablet_manager.cc:1367] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:13.798775 11704 consensus_queue.cc:260] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.799047 11704 ts_tablet_manager.cc:1398] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.799116 11706 raft_consensus.cc:348] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.799214 11706 raft_consensus.cc:374] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.799243 11706 raft_consensus.cc:729] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.799346 11703 tablet_bootstrap.cc:492] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:13.799422 11703 ts_tablet_manager.cc:1367] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:13.799358 11706 consensus_queue.cc:260] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.799649 11706 ts_tablet_manager.cc:1398] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.799858 11703 raft_consensus.cc:348] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.799964 11703 raft_consensus.cc:374] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.799995 11703 raft_consensus.cc:729] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.800101 11703 consensus_queue.cc:260] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.800361 11703 ts_tablet_manager.cc:1398] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:13.800514 11703 tablet_bootstrap.cc:492] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:13.801416 11703 tablet_bootstrap.cc:654] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.802434 11703 tablet_bootstrap.cc:492] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:13.802520 11703 ts_tablet_manager.cc:1367] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:01:13.802963 11703 raft_consensus.cc:348] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.803076 11703 raft_consensus.cc:374] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.803134 11703 raft_consensus.cc:729] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.803258 11703 consensus_queue.cc:260] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.803512 11703 ts_tablet_manager.cc:1398] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:13.815742 11684 raft_consensus.cc:482] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:13.815893 11684 raft_consensus.cc:504] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.816288 11684 leader_election.cc:288] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:13.816659 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1b9c544e23474791c93f4094b4c672" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:13.816721 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1b9c544e23474791c93f4094b4c672" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:13.816794 10506 raft_consensus.cc:2454] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:13.816867 10112 raft_consensus.cc:2454] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 0.
I20240502 14:01:13.817099 10180 leader_election.cc:302] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:13.817266 11684 raft_consensus.cc:2781] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:13.817322 11684 raft_consensus.cc:482] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:13.817366 11684 raft_consensus.cc:3037] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.818207 11684 raft_consensus.cc:504] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.818500 11684 leader_election.cc:288] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:13.818786 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1b9c544e23474791c93f4094b4c672" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:13.818873 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1b9c544e23474791c93f4094b4c672" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:13.818904 10506 raft_consensus.cc:3037] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.818990 10112 raft_consensus.cc:3037] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.819911 10112 raft_consensus.cc:2454] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:13.819918 10506 raft_consensus.cc:2454] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 1.
I20240502 14:01:13.820195 10180 leader_election.cc:302] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:13.820333 11684 raft_consensus.cc:2781] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:13.820425 11684 raft_consensus.cc:686] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:13.820561 11684 consensus_queue.cc:237] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.822335 9964 catalog_manager.cc:5484] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 1 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:13.832990 11247 raft_consensus.cc:482] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:13.833112 11247 raft_consensus.cc:504] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.833518 11247 leader_election.cc:288] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:13.833948 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0577d6cbf5a04722843d7ecf9cf28be9" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be" is_pre_election: true
I20240502 14:01:13.833977 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0577d6cbf5a04722843d7ecf9cf28be9" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:13.834089 10112 raft_consensus.cc:2454] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:13.834112 10243 raft_consensus.cc:2454] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 0.
I20240502 14:01:13.834380 10311 leader_election.cc:302] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:13.834554 11247 raft_consensus.cc:2781] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:13.834609 11247 raft_consensus.cc:482] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:13.834662 11247 raft_consensus.cc:3037] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.835459 11247 raft_consensus.cc:504] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.835757 11247 leader_election.cc:288] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:13.836163 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0577d6cbf5a04722843d7ecf9cf28be9" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:13.836272 10112 raft_consensus.cc:3037] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.836220 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0577d6cbf5a04722843d7ecf9cf28be9" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:13.836356 10243 raft_consensus.cc:3037] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.837036 10112 raft_consensus.cc:2454] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:13.837198 10243 raft_consensus.cc:2454] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 1.
I20240502 14:01:13.837316 10311 leader_election.cc:302] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:13.837491 11247 raft_consensus.cc:2781] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:13.837687 11247 raft_consensus.cc:686] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:13.837831 11247 consensus_queue.cc:237] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:13.839570 9964 catalog_manager.cc:5484] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 1 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:13.984172 11595 raft_consensus.cc:482] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:13.984304 11595 raft_consensus.cc:504] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.984761 11595 leader_election.cc:288] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:13.985221 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c958a9027a064dcb9026608541a22883" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" is_pre_election: true
I20240502 14:01:13.985277 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c958a9027a064dcb9026608541a22883" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:13.985394 10506 raft_consensus.cc:2454] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:13.985405 10375 raft_consensus.cc:2454] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:13.985713 10049 leader_election.cc:302] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:13.985915 11595 raft_consensus.cc:2781] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:13.985988 11595 raft_consensus.cc:482] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:13.986016 11595 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.986819 11595 raft_consensus.cc:504] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.987162 11595 leader_election.cc:288] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:13.987531 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c958a9027a064dcb9026608541a22883" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:13.987530 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c958a9027a064dcb9026608541a22883" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:13.987663 10506 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.987663 10375 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.988668 10375 raft_consensus.cc:2454] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:13.988668 10506 raft_consensus.cc:2454] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:13.989008 10049 leader_election.cc:302] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:13.989185 11595 raft_consensus.cc:2781] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:13.989279 11595 raft_consensus.cc:686] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Leader. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:13.989454 11595 consensus_queue.cc:237] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:13.992369 9969 catalog_manager.cc:5484] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be reported cstate change: term changed from 0 to 1, leader changed from <none> to 068f58913ea3478599df829d5118f7be (127.26.25.129). New cstate: current_term: 1 leader_uuid: "068f58913ea3478599df829d5118f7be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:14.139197 11720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:14.139429 11720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:14.142784 11720 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240502 14:01:14.243515 10375 raft_consensus.cc:1261] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.243971 11596 consensus_queue.cc:1035] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.248930 10243 raft_consensus.cc:1261] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.248975 10114 raft_consensus.cc:1261] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.249550 10506 raft_consensus.cc:1261] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.249554 11247 consensus_queue.cc:1035] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.252439 11595 consensus_queue.cc:1035] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.252704 11247 consensus_queue.cc:1035] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.253232 11684 consensus_queue.cc:1035] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.260275 10508 raft_consensus.cc:1261] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.261126 11684 consensus_queue.cc:1035] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 123.56 ms
time per row: 0.0617798 ms
I20240502 14:01:14.313727 9963 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44294:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:01:14.319718 10355 tablet_service.cc:1467] Processing CreateTablet for tablet e5936fe6a561440281de30e976d12fb3 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4698217f20fa4399b25552cfcfeb051a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.319772 10486 tablet_service.cc:1467] Processing CreateTablet for tablet e5936fe6a561440281de30e976d12fb3 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4698217f20fa4399b25552cfcfeb051a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.319772 10225 tablet_service.cc:1467] Processing CreateTablet for tablet e5936fe6a561440281de30e976d12fb3 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4698217f20fa4399b25552cfcfeb051a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.320108 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5936fe6a561440281de30e976d12fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.320108 10225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5936fe6a561440281de30e976d12fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.320111 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5936fe6a561440281de30e976d12fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.320194 10223 tablet_service.cc:1467] Processing CreateTablet for tablet 35c205ff5aaf46c09fc93b9ded95e971 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4698217f20fa4399b25552cfcfeb051a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.320428 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35c205ff5aaf46c09fc93b9ded95e971. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.322855 10223 tablet_service.cc:1467] Processing CreateTablet for tablet fe234bfab79843d79b0ce5228e678ab8 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4698217f20fa4399b25552cfcfeb051a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.322971 11779 tablet_bootstrap.cc:492] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:14.322950 10354 tablet_service.cc:1467] Processing CreateTablet for tablet fe234bfab79843d79b0ce5228e678ab8 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4698217f20fa4399b25552cfcfeb051a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.323073 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe234bfab79843d79b0ce5228e678ab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.323151 10354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe234bfab79843d79b0ce5228e678ab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.323221 10093 tablet_service.cc:1467] Processing CreateTablet for tablet fe234bfab79843d79b0ce5228e678ab8 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4698217f20fa4399b25552cfcfeb051a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.323473 10093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe234bfab79843d79b0ce5228e678ab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.323951 11779 tablet_bootstrap.cc:654] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.324793 10486 tablet_service.cc:1467] Processing CreateTablet for tablet 35c205ff5aaf46c09fc93b9ded95e971 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4698217f20fa4399b25552cfcfeb051a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.325003 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35c205ff5aaf46c09fc93b9ded95e971. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.325207 10355 tablet_service.cc:1467] Processing CreateTablet for tablet 35c205ff5aaf46c09fc93b9ded95e971 (DEFAULT_TABLE table=rebalance_test_table_9 [id=4698217f20fa4399b25552cfcfeb051a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.325417 10355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35c205ff5aaf46c09fc93b9ded95e971. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.327023 11777 tablet_bootstrap.cc:492] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:14.327577 11781 tablet_bootstrap.cc:492] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be: Bootstrap starting.
I20240502 14:01:14.327997 11779 tablet_bootstrap.cc:492] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:14.328080 11779 ts_tablet_manager.cc:1367] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 14:01:14.328132 11777 tablet_bootstrap.cc:654] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.328500 11781 tablet_bootstrap.cc:654] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.328735 11778 tablet_bootstrap.cc:492] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:14.328951 11779 raft_consensus.cc:348] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.329046 11779 raft_consensus.cc:374] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.329082 11779 raft_consensus.cc:729] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.329231 11779 consensus_queue.cc:260] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.329591 11779 ts_tablet_manager.cc:1398] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.329654 11778 tablet_bootstrap.cc:654] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.329878 11779 tablet_bootstrap.cc:492] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:14.329885 11777 tablet_bootstrap.cc:492] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:14.330016 11777 ts_tablet_manager.cc:1367] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:01:14.330624 11778 tablet_bootstrap.cc:492] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:14.330662 11777 raft_consensus.cc:348] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.330754 11777 raft_consensus.cc:374] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.330781 11777 raft_consensus.cc:729] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.330943 11777 consensus_queue.cc:260] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.332474 11779 tablet_bootstrap.cc:654] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.332592 11777 ts_tablet_manager.cc:1398] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:01:14.332779 11777 tablet_bootstrap.cc:492] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:14.332937 11778 ts_tablet_manager.cc:1367] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:14.333369 11778 raft_consensus.cc:348] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.333485 11778 raft_consensus.cc:374] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.333519 11778 raft_consensus.cc:729] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.333631 11778 consensus_queue.cc:260] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.333909 11778 ts_tablet_manager.cc:1398] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.334057 11778 tablet_bootstrap.cc:492] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Bootstrap starting.
I20240502 14:01:14.334098 11781 tablet_bootstrap.cc:492] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be: No bootstrap required, opened a new log
I20240502 14:01:14.334177 11781 ts_tablet_manager.cc:1367] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:01:14.334609 11781 raft_consensus.cc:348] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:14.334712 11781 raft_consensus.cc:374] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.334782 11781 raft_consensus.cc:729] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.334863 11778 tablet_bootstrap.cc:654] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.334939 11781 consensus_queue.cc:260] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:14.335259 11781 ts_tablet_manager.cc:1398] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.336114 11778 tablet_bootstrap.cc:492] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: No bootstrap required, opened a new log
I20240502 14:01:14.336292 11778 ts_tablet_manager.cc:1367] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:14.336995 11778 raft_consensus.cc:348] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.337157 11778 raft_consensus.cc:374] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.337240 11778 raft_consensus.cc:729] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.337802 11778 consensus_queue.cc:260] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.338099 11778 ts_tablet_manager.cc:1398] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:14.339191 11777 tablet_bootstrap.cc:654] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.341799 11777 tablet_bootstrap.cc:492] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:14.341886 11777 ts_tablet_manager.cc:1367] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240502 14:01:14.342236 11777 raft_consensus.cc:348] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.342331 11777 raft_consensus.cc:374] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.342353 11777 raft_consensus.cc:729] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.342461 11777 consensus_queue.cc:260] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.342713 11777 ts_tablet_manager.cc:1398] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.342864 11777 tablet_bootstrap.cc:492] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc: Bootstrap starting.
I20240502 14:01:14.343768 11777 tablet_bootstrap.cc:654] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.344775 11779 tablet_bootstrap.cc:492] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:14.344902 11779 ts_tablet_manager.cc:1367] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.001s
I20240502 14:01:14.345431 11779 raft_consensus.cc:348] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:14.345563 11779 raft_consensus.cc:374] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.345597 11779 raft_consensus.cc:729] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.345839 11779 consensus_queue.cc:260] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:14.346336 11779 ts_tablet_manager.cc:1398] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 14:01:14.346510 11779 tablet_bootstrap.cc:492] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51: Bootstrap starting.
I20240502 14:01:14.347635 11779 tablet_bootstrap.cc:654] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.348656 11779 tablet_bootstrap.cc:492] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51: No bootstrap required, opened a new log
I20240502 14:01:14.348740 11779 ts_tablet_manager.cc:1367] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:14.349078 11779 raft_consensus.cc:348] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.349162 11779 raft_consensus.cc:374] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.349195 11779 raft_consensus.cc:729] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.349324 11779 consensus_queue.cc:260] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.349539 11779 ts_tablet_manager.cc:1398] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.349828 11777 tablet_bootstrap.cc:492] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc: No bootstrap required, opened a new log
I20240502 14:01:14.349911 11777 ts_tablet_manager.cc:1367] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:01:14.350248 11777 raft_consensus.cc:348] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:14.350355 11777 raft_consensus.cc:374] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.350390 11777 raft_consensus.cc:729] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.350520 11777 consensus_queue.cc:260] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:14.350788 11777 ts_tablet_manager.cc:1398] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.351132 11130 raft_consensus.cc:482] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:14.351243 11130 raft_consensus.cc:504] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.351681 11130 leader_election.cc:288] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:14.352073 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c205ff5aaf46c09fc93b9ded95e971" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:14.352097 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c205ff5aaf46c09fc93b9ded95e971" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:14.352200 10375 raft_consensus.cc:2454] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 0.
I20240502 14:01:14.352200 10243 raft_consensus.cc:2454] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 0.
I20240502 14:01:14.352504 10442 leader_election.cc:302] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:14.352645 11130 raft_consensus.cc:2781] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:14.352694 11130 raft_consensus.cc:482] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:14.352730 11130 raft_consensus.cc:3037] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.353413 11130 raft_consensus.cc:504] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.353705 11130 leader_election.cc:288] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:14.353984 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c205ff5aaf46c09fc93b9ded95e971" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:14.354094 10243 raft_consensus.cc:3037] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.354068 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c205ff5aaf46c09fc93b9ded95e971" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:14.354167 10375 raft_consensus.cc:3037] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.354976 10375 raft_consensus.cc:2454] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 1.
I20240502 14:01:14.355020 10243 raft_consensus.cc:2454] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 1.
I20240502 14:01:14.355284 10440 leader_election.cc:302] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:14.355495 11130 raft_consensus.cc:2781] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:14.355700 11130 raft_consensus.cc:686] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:14.355887 11130 consensus_queue.cc:237] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.357319 9963 catalog_manager.cc:5484] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:14.491040 11596 raft_consensus.cc:482] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:14.491166 11596 raft_consensus.cc:504] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:14.491526 11596 leader_election.cc:288] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:14.491952 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:14.491972 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:14.492086 10375 raft_consensus.cc:2454] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:14.492134 10243 raft_consensus.cc:2454] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 0.
I20240502 14:01:14.492370 10047 leader_election.cc:302] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:14.492547 11596 raft_consensus.cc:2781] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:14.492607 11596 raft_consensus.cc:482] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:14.492645 11596 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.493507 11596 raft_consensus.cc:504] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:14.493872 11596 leader_election.cc:288] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:14.494129 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:14.494254 10243 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.494271 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:14.494401 10375 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.495087 10243 raft_consensus.cc:2454] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:14.495177 10375 raft_consensus.cc:2454] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 1.
I20240502 14:01:14.495398 10049 leader_election.cc:302] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:14.495551 11596 raft_consensus.cc:2781] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:14.495644 11596 raft_consensus.cc:686] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Leader. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:14.495788 11596 consensus_queue.cc:237] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:14.497695 9963 catalog_manager.cc:5484] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be reported cstate change: term changed from 0 to 1, leader changed from <none> to 068f58913ea3478599df829d5118f7be (127.26.25.129). New cstate: current_term: 1 leader_uuid: "068f58913ea3478599df829d5118f7be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:14.580024 11130 raft_consensus.cc:482] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:14.580152 11130 raft_consensus.cc:504] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.580588 11130 leader_election.cc:288] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:14.581017 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5936fe6a561440281de30e976d12fb3" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc" is_pre_election: true
I20240502 14:01:14.581020 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5936fe6a561440281de30e976d12fb3" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51" is_pre_election: true
I20240502 14:01:14.581162 10243 raft_consensus.cc:2454] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 0.
I20240502 14:01:14.581182 10375 raft_consensus.cc:2454] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 0.
I20240502 14:01:14.581470 10440 leader_election.cc:302] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:14.581640 11130 raft_consensus.cc:2781] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:14.581698 11130 raft_consensus.cc:482] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:14.581725 11130 raft_consensus.cc:3037] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.582510 11130 raft_consensus.cc:504] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.582823 11130 leader_election.cc:288] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:14.583151 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5936fe6a561440281de30e976d12fb3" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:14.583149 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5936fe6a561440281de30e976d12fb3" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:14.583250 10375 raft_consensus.cc:3037] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.583273 10243 raft_consensus.cc:3037] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.584030 10375 raft_consensus.cc:2454] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 1.
I20240502 14:01:14.584127 10243 raft_consensus.cc:2454] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 1.
I20240502 14:01:14.584276 10440 leader_election.cc:302] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:14.584442 11130 raft_consensus.cc:2781] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:14.584519 11130 raft_consensus.cc:686] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:14.584662 11130 consensus_queue.cc:237] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:14.587019 9963 catalog_manager.cc:5484] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 1 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } }
W20240502 14:01:14.730852 11794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:14.731170 11794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:14.734334 11794 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240502 14:01:14.809496 11130 consensus_queue.cc:1035] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.813592 11790 consensus_queue.cc:1035] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.832008 10243 raft_consensus.cc:1261] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.832113 10375 raft_consensus.cc:1261] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer 068f58913ea3478599df829d5118f7be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.833158 11596 consensus_queue.cc:1035] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.834321 11596 consensus_queue.cc:1035] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.836617 10245 raft_consensus.cc:1261] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.838057 11790 consensus_queue.cc:1035] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.842056 10375 raft_consensus.cc:1261] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.846192 11130 consensus_queue.cc:1035] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 114.625 ms
time per row: 0.0573126 ms
I20240502 14:01:14.900027 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.133:42411
--local_ip_for_outbound_sockets=127.26.25.133
--tserver_master_addrs=127.26.25.190:39653
--webserver_port=35167
--webserver_interface=127.26.25.133
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:01:15.017517 11851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:15.017740 11851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:15.017782 11851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:15.020962 11851 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:15.021032 11851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:15.021098 11851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.133
I20240502 14:01:15.024926 11851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.133:42411
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.26.25.133
--webserver_port=35167
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.11851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.133
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:15.025843 11851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:15.026934 11851 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240502 14:01:15.036265 11851 server_base.cc:1034] running on GCE node
W20240502 14:01:15.036779 11857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:15.037627 11856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:15.038368 11859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:15.038800 11851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:15.039392 11851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:15.040551 11851 hybrid_clock.cc:648] HybridClock initialized: now 1714658475040528 us; error 39 us; skew 500 ppm
I20240502 14:01:15.042693 11851 webserver.cc:460] Webserver started at http://127.26.25.133:35167/ using document root <none> and password file <none>
I20240502 14:01:15.043274 11851 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:15.043355 11851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:15.047201 11851 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:15.049127 11865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.050125 11851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.050240 11851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "5a19a740e4ae4be8955282aced7210f5"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:15.050662 11851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:15.076225 11851 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:15.076853 11851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:15.077059 11851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:15.077630 11851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:15.078647 11851 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:15.078697 11851 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.078756 11851 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:15.078784 11851 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.088543 11851 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.133:42411
I20240502 14:01:15.088564 11978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.133:42411 every 8 connection(s)
I20240502 14:01:15.089713 11851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 14:01:15.096632 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 11851
I20240502 14:01:15.097110 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.134:34253
--local_ip_for_outbound_sockets=127.26.25.134
--tserver_master_addrs=127.26.25.190:39653
--webserver_port=34949
--webserver_interface=127.26.25.134
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:15.099417 11979 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:15.099642 11979 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:15.100220 11979 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:15.101483 9971 ts_manager.cc:194] Re-registered known tserver with Master: 5a19a740e4ae4be8955282aced7210f5 (127.26.25.133:42411)
I20240502 14:01:15.102339 9971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.133:45297
W20240502 14:01:15.211624 11982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:15.211890 11982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:15.211962 11982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:15.215015 11982 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:15.215082 11982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:15.215175 11982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.134
I20240502 14:01:15.218930 11982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.134:34253
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.26.25.134
--webserver_port=34949
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.11982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.134
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:15.219871 11982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:15.220965 11982 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:15.229709 11987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:15.230307 11988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:15.231200 11990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:15.231922 11982 server_base.cc:1034] running on GCE node
I20240502 14:01:15.232358 11982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:15.233036 11982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:15.234190 11982 hybrid_clock.cc:648] HybridClock initialized: now 1714658475234135 us; error 74 us; skew 500 ppm
I20240502 14:01:15.236227 11982 webserver.cc:460] Webserver started at http://127.26.25.134:34949/ using document root <none> and password file <none>
I20240502 14:01:15.236912 11982 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:15.237000 11982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:15.240531 11982 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:01:15.242547 11996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.243624 11982 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:15.243717 11982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "f1bfaf3d95f54984a2b1f97a23352440"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:15.244151 11982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:15.266834 11982 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:15.267390 11982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:15.267544 11982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:15.268108 11982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:15.268952 11982 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:15.268998 11982 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.269045 11982 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:15.269076 11982 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.277443 11982 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.134:34253
I20240502 14:01:15.277613 12109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.134:34253 every 8 connection(s)
I20240502 14:01:15.278499 11982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 14:01:15.283766 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 11982
I20240502 14:01:15.284327 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.135:38811
--local_ip_for_outbound_sockets=127.26.25.135
--tserver_master_addrs=127.26.25.190:39653
--webserver_port=33039
--webserver_interface=127.26.25.135
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:15.288638 12110 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:15.288872 12110 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:15.289407 12110 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:15.290450 9971 ts_manager.cc:194] Re-registered known tserver with Master: f1bfaf3d95f54984a2b1f97a23352440 (127.26.25.134:34253)
I20240502 14:01:15.291039 9971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.134:34441
W20240502 14:01:15.398334 12113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:15.398555 12113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:15.398602 12113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:15.401696 12113 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:15.401765 12113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:15.401870 12113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.135
I20240502 14:01:15.405808 12113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.135:38811
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.26.25.135
--webserver_port=33039
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.12113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.135
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:15.406713 12113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:15.407899 12113 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:15.417214 12119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:15.417893 12122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:15.418587 12120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:15.419260 12113 server_base.cc:1034] running on GCE node
I20240502 14:01:15.419672 12113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:15.420327 12113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:15.421514 12113 hybrid_clock.cc:648] HybridClock initialized: now 1714658475421489 us; error 32 us; skew 500 ppm
I20240502 14:01:15.423418 12113 webserver.cc:460] Webserver started at http://127.26.25.135:33039/ using document root <none> and password file <none>
I20240502 14:01:15.424088 12113 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:15.424187 12113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:15.428331 12113 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:01:15.430235 12129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.431357 12113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:15.431469 12113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "e3d80b4f40cf438d9d0a63f54a1f5907"
format_stamp: "Formatted at 2024-05-02 14:01:04 on dist-test-slave-sldb"
I20240502 14:01:15.431922 12113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:15.447522 12113 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:15.448177 12113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:15.448339 12113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:15.448875 12113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:15.449764 12113 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:15.449813 12113 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.449875 12113 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:15.449900 12113 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.459059 12113 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.135:38811
I20240502 14:01:15.459095 12242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.135:38811 every 8 connection(s)
I20240502 14:01:15.460270 12113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 14:01:15.461215 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 12113
I20240502 14:01:15.461601 26726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
/tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.25.136:34019
--local_ip_for_outbound_sockets=127.26.25.136
--tserver_master_addrs=127.26.25.190:39653
--webserver_port=43337
--webserver_interface=127.26.25.136
--builtin_ntp_servers=127.26.25.148:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:15.470239 12243 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:15.470504 12243 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:15.471148 12243 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:15.472258 9971 ts_manager.cc:194] Re-registered known tserver with Master: e3d80b4f40cf438d9d0a63f54a1f5907 (127.26.25.135:38811)
I20240502 14:01:15.472815 9971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.135:45697
W20240502 14:01:15.572597 12245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:15.572822 12245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:15.572867 12245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:15.576009 12245 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:15.576086 12245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:15.576169 12245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.25.136
I20240502 14:01:15.580056 12245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.25.148:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.25.136:34019
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.26.25.136
--webserver_port=43337
--tserver_master_addrs=127.26.25.190:39653
--never_fsync=true
--heap_profile_path=/tmp/kudu.12245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.25.136
--log_dir=/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:15.580967 12245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:15.582072 12245 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:15.590727 12251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:15.591333 12245 server_base.cc:1034] running on GCE node
W20240502 14:01:15.592144 12254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:15.592880 12252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:15.593309 12245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:15.593896 12245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:15.595075 12245 hybrid_clock.cc:648] HybridClock initialized: now 1714658475595059 us; error 33 us; skew 500 ppm
I20240502 14:01:15.597203 12245 webserver.cc:460] Webserver started at http://127.26.25.136:43337/ using document root <none> and password file <none>
I20240502 14:01:15.597832 12245 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:15.597929 12245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:15.601298 12245 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:01:15.603099 12260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.604092 12245 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:15.604192 12245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "1ca6438d01864c25bf5eb0a54955e8a9"
format_stamp: "Formatted at 2024-05-02 14:01:05 on dist-test-slave-sldb"
I20240502 14:01:15.604594 12245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:15.627547 12245 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:15.628170 12245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:15.628329 12245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:15.628844 12245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:15.629801 12245 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:15.629850 12245 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.629907 12245 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:15.629947 12245 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:15.638824 12245 rpc_server.cc:307] RPC server started. Bound to: 127.26.25.136:34019
I20240502 14:01:15.638865 12373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.25.136:34019 every 8 connection(s)
I20240502 14:01:15.639809 12245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskW3Zz6D/test-tmp/rebalancer_tool-test.1.RebalancingDuringElectionStormTest.RoundRobin_1.1714658379150170-26726-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 14:01:15.648193 26726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskW3Zz6D/build/debug/bin/kudu as pid 12245
I20240502 14:01:15.648442 26726 test_util.cc:268] Using random seed: 156902841
I20240502 14:01:15.648818 12374 heartbeater.cc:347] Connected to a master server at 127.26.25.190:39653
I20240502 14:01:15.649071 12374 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:15.649689 12374 heartbeater.cc:511] Master 127.26.25.190:39653 requested a full tablet report, sending...
I20240502 14:01:15.650877 9971 ts_manager.cc:194] Re-registered known tserver with Master: 1ca6438d01864c25bf5eb0a54955e8a9 (127.26.25.136:34019)
I20240502 14:01:15.651536 9971 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.25.136:42339
I20240502 14:01:15.659508 26726 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_0 already exists
I20240502 14:01:15.659745 26726 test_util.cc:268] Using random seed: 156914142
I20240502 14:01:15.670097 26726 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_1 already exists
I20240502 14:01:15.670287 26726 test_util.cc:268] Using random seed: 156924687
I20240502 14:01:15.680073 10508 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "35c205ff5aaf46c09fc93b9ded95e971"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.680274 10508 raft_consensus.cc:465] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 14:01:15.680964 26726 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_2 already exists
I20240502 14:01:15.681115 26726 test_util.cc:268] Using random seed: 156935513
I20240502 14:01:15.682539 10508 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.682672 10508 raft_consensus.cc:482] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.682737 10508 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.683604 10508 raft_consensus.cc:504] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.684036 10508 leader_election.cc:288] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:15.684497 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.684620 10243 raft_consensus.cc:3032] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.684671 10243 raft_consensus.cc:729] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:15.684830 10243 consensus_queue.cc:260] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.684938 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:15.685043 10375 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.685120 10243 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.686014 10243 raft_consensus.cc:2454] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.686015 10375 raft_consensus.cc:2454] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.686323 10442 leader_election.cc:302] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.686642 11836 raft_consensus.cc:2781] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.686861 11836 raft_consensus.cc:686] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.687005 11836 consensus_queue.cc:237] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.690078 9966 catalog_manager.cc:5484] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.696770 10508 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2379dbd40b744991be25c2914f878913"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.696897 10508 raft_consensus.cc:465] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 14:01:15.697919 10508 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.698042 10508 raft_consensus.cc:482] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.698112 10508 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.699287 10508 raft_consensus.cc:504] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.699805 10508 leader_election.cc:288] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:15.700620 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:15.700646 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.700747 10243 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.700747 10375 raft_consensus.cc:3032] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.700793 10375 raft_consensus.cc:729] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.700973 10375 consensus_queue.cc:260] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.701246 10375 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.701751 10243 raft_consensus.cc:2454] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.700551 10508 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "45f819d4fb5946248f5ec084d252bcd0"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.702013 10508 raft_consensus.cc:482] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.702080 10508 raft_consensus.cc:3037] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.702275 10375 raft_consensus.cc:2454] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.702289 26726 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_3 already exists
I20240502 14:01:15.702464 26726 test_util.cc:268] Using random seed: 156956862
I20240502 14:01:15.702487 10442 leader_election.cc:302] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.702656 11836 raft_consensus.cc:2781] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.702733 11836 raft_consensus.cc:686] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.702863 11836 consensus_queue.cc:237] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.703119 10508 raft_consensus.cc:504] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.703840 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f819d4fb5946248f5ec084d252bcd0" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.703998 10114 raft_consensus.cc:3032] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.704049 10114 raft_consensus.cc:729] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:15.704201 10114 consensus_queue.cc:260] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.704536 10114 raft_consensus.cc:3037] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.705607 10114 raft_consensus.cc:2454] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.707631 10442 leader_election.cc:302] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.708175 11836 raft_consensus.cc:2781] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.708264 11836 raft_consensus.cc:686] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.708389 11836 consensus_queue.cc:237] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.709223 9971 catalog_manager.cc:5484] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:15.710917 10508 leader_election.cc:288] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:15.713008 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f819d4fb5946248f5ec084d252bcd0" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:15.713137 10376 raft_consensus.cc:3037] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.714030 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c958a9027a064dcb9026608541a22883"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.714135 10507 raft_consensus.cc:482] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.714141 10376 raft_consensus.cc:2454] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.714188 10507 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.715205 10507 raft_consensus.cc:504] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
W20240502 14:01:15.715337 10336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54300: Illegal state: replica 243b53aabd2d4443a8971e707bcc4a51 is not leader of this config: current role FOLLOWER
I20240502 14:01:15.715626 10507 leader_election.cc:288] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:15.716281 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c958a9027a064dcb9026608541a22883" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.716408 10112 raft_consensus.cc:3032] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.716468 10112 raft_consensus.cc:729] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:15.716553 9966 catalog_manager.cc:5484] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 068f58913ea3478599df829d5118f7be (127.26.25.129) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.716619 10112 consensus_queue.cc:260] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.716812 10112 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.717725 10112 raft_consensus.cc:2454] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.718287 26726 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_4 already exists
I20240502 14:01:15.718454 26726 test_util.cc:268] Using random seed: 156972852
I20240502 14:01:15.720067 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c958a9027a064dcb9026608541a22883" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:15.720182 10376 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.720572 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.720691 10507 raft_consensus.cc:482] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.720748 10507 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.721072 10442 leader_election.cc:302] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.721127 10376 raft_consensus.cc:2454] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.721750 12417 raft_consensus.cc:2781] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.721922 12417 raft_consensus.cc:686] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
W20240502 14:01:15.721905 10202 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51968: Illegal state: replica 1ca5103abf454aaca375e9e07eb139cc is not leader of this config: current role FOLLOWER
I20240502 14:01:15.722056 12417 consensus_queue.cc:237] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.721683 10507 raft_consensus.cc:504] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.723592 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:15.723702 10376 raft_consensus.cc:3032] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.723752 10376 raft_consensus.cc:729] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.723904 10376 consensus_queue.cc:260] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.724077 10376 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.724988 10376 raft_consensus.cc:2454] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.725271 10440 leader_election.cc:302] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.725670 11836 raft_consensus.cc:2781] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.725746 11836 raft_consensus.cc:686] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.725853 11836 consensus_queue.cc:237] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.725303 10507 leader_election.cc:288] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:15.726368 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.726500 10243 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.726779 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a804b61491304055865347e70e8a9778"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.726889 10507 raft_consensus.cc:482] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.726954 10507 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.727430 10243 raft_consensus.cc:2454] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.728572 10507 raft_consensus.cc:504] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.728946 10507 leader_election.cc:288] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:15.729296 9966 catalog_manager.cc:5484] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 068f58913ea3478599df829d5118f7be (127.26.25.129) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:15.729585 9966 catalog_manager.cc:5484] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:15.729840 10336 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54318: Illegal state: replica 243b53aabd2d4443a8971e707bcc4a51 is not leader of this config: current role FOLLOWER
I20240502 14:01:15.730573 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a804b61491304055865347e70e8a9778" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:15.730690 10375 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.731623 10375 raft_consensus.cc:2454] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.732072 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a804b61491304055865347e70e8a9778" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.732232 10243 raft_consensus.cc:3032] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.732275 10243 raft_consensus.cc:729] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:15.732411 10243 consensus_queue.cc:260] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.732602 10243 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.732929 10440 leader_election.cc:302] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.733547 10243 raft_consensus.cc:2454] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.733841 11836 raft_consensus.cc:2781] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.734037 11836 raft_consensus.cc:686] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.734169 11836 consensus_queue.cc:237] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.735116 10376 raft_consensus.cc:1261] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.735589 11836 consensus_queue.cc:1035] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.735905 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c211d47d28ae45ed9bdf0471b553edb1"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.736017 10507 raft_consensus.cc:482] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.736078 10507 raft_consensus.cc:3037] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.737072 10507 raft_consensus.cc:504] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:15.738025 9966 catalog_manager.cc:5484] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.738610 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c211d47d28ae45ed9bdf0471b553edb1" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.738719 10245 raft_consensus.cc:3032] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.738765 10245 raft_consensus.cc:729] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:15.738885 10245 consensus_queue.cc:260] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:15.739059 10245 raft_consensus.cc:3037] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.739027 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c211d47d28ae45ed9bdf0471b553edb1" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.739149 10112 raft_consensus.cc:3037] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.740211 10245 raft_consensus.cc:2454] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.740192 10112 raft_consensus.cc:2454] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.740537 10442 leader_election.cc:302] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.741050 12417 raft_consensus.cc:2781] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.741142 12417 raft_consensus.cc:686] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.741271 12417 consensus_queue.cc:237] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:15.742525 10507 leader_election.cc:288] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:15.742543 10243 raft_consensus.cc:1261] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.743407 11836 consensus_queue.cc:1035] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.743794 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.743903 10507 raft_consensus.cc:482] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.743952 10507 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.744868 9971 catalog_manager.cc:5484] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.745049 10507 raft_consensus.cc:504] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.745635 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.745744 10245 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.745765 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:15.745870 10376 raft_consensus.cc:3032] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.745911 10376 raft_consensus.cc:729] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.746042 10376 consensus_queue.cc:260] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.746212 10376 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.746253 10375 raft_consensus.cc:1261] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.746639 11836 consensus_queue.cc:1035] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.746716 10245 raft_consensus.cc:2454] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.746976 10442 leader_election.cc:302] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.747124 10376 raft_consensus.cc:2454] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.747182 10245 raft_consensus.cc:1261] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.747459 11836 raft_consensus.cc:2781] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.747538 11836 raft_consensus.cc:686] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.747591 12417 consensus_queue.cc:1035] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.747642 11836 consensus_queue.cc:237] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
W20240502 14:01:15.750811 10202 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51968: Illegal state: replica 1ca5103abf454aaca375e9e07eb139cc is not leader of this config: current role FOLLOWER
I20240502 14:01:15.750864 10507 leader_election.cc:288] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:15.751713 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.751864 10507 raft_consensus.cc:482] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.751929 10507 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.754460 10507 raft_consensus.cc:504] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.756721 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.756857 10245 raft_consensus.cc:3032] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.756906 10245 raft_consensus.cc:729] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:15.757040 10245 consensus_queue.cc:260] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.757206 10245 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:01:15.757380 10335 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54300: Illegal state: replica 243b53aabd2d4443a8971e707bcc4a51 is not leader of this config: current role FOLLOWER
I20240502 14:01:15.757973 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.758095 10114 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.758311 10245 raft_consensus.cc:2454] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.758567 10442 leader_election.cc:302] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.758822 11836 raft_consensus.cc:2781] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.758925 11836 raft_consensus.cc:686] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.758885 9963 catalog_manager.cc:5484] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:15.759061 11836 consensus_queue.cc:237] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.759153 10114 raft_consensus.cc:2454] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.760078 10507 leader_election.cc:288] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:15.760735 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e5936fe6a561440281de30e976d12fb3"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.760838 10507 raft_consensus.cc:465] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 14:01:15.761853 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8b1dff0b06824632869c970e3a031d3b"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.761937 10507 raft_consensus.cc:465] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 14:01:15.762486 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9a76fe53d2dd4b12bc04ac52089a7120"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.762570 10507 raft_consensus.cc:482] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.762620 10507 raft_consensus.cc:3037] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.765264 10245 raft_consensus.cc:1261] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.765686 10376 raft_consensus.cc:1261] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.765720 12417 consensus_queue.cc:1035] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.766058 12417 consensus_queue.cc:1035] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.768332 10507 raft_consensus.cc:504] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.768920 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a76fe53d2dd4b12bc04ac52089a7120" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.769042 10112 raft_consensus.cc:3037] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.769996 10112 raft_consensus.cc:2454] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.770714 10442 leader_election.cc:302] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
W20240502 14:01:15.770727 10073 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39034: Illegal state: replica 068f58913ea3478599df829d5118f7be is not leader of this config: current role FOLLOWER
I20240502 14:01:15.770895 11836 raft_consensus.cc:2781] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.770972 11836 raft_consensus.cc:686] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.771087 11836 consensus_queue.cc:237] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.771752 10507 leader_election.cc:288] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:15.770270 9966 catalog_manager.cc:5484] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:15.772805 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a76fe53d2dd4b12bc04ac52089a7120" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:15.773893 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "67cf72901dd0434d9b88902343d45977"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.774749 10374 raft_consensus.cc:3032] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.774883 10374 raft_consensus.cc:729] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.775250 10374 consensus_queue.cc:260] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.775722 10374 raft_consensus.cc:3037] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:01:15.776547 12452 consensus_queue.cc:1159] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" responder_term: 1 status { last_received { term: 1 index: 6 } last_committed_idx: 5 last_received_current_leader { term: 1 index: 6 } }
I20240502 14:01:15.777788 10374 raft_consensus.cc:2454] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.778344 10507 raft_consensus.cc:482] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.778435 10507 raft_consensus.cc:3037] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.779553 12428 mvcc.cc:205] Tried to move back new op lower bound from 7023241116731244544 to 7023241116608737280. Current Snapshot: MvccSnapshot[applied={T|T < 7023241116731244544}]
I20240502 14:01:15.779790 10507 raft_consensus.cc:504] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.782124 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cf72901dd0434d9b88902343d45977" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.782250 10112 raft_consensus.cc:3037] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.783248 10112 raft_consensus.cc:2454] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.783730 10442 leader_election.cc:302] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.785032 12417 raft_consensus.cc:2781] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.785120 12417 raft_consensus.cc:686] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.785252 12417 consensus_queue.cc:237] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.787354 10114 raft_consensus.cc:1261] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.787729 12458 consensus_queue.cc:1035] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.788753 10374 raft_consensus.cc:1261] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.790199 12417 consensus_queue.cc:1035] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.796756 10507 leader_election.cc:288] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:15.797751 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3474ecec1b6f4fc29d6db00b8fa56b06"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.797883 10507 raft_consensus.cc:465] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 14:01:15.798893 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.799122 10507 raft_consensus.cc:482] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.799206 10507 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.800400 10507 raft_consensus.cc:504] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.801247 26726 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_5 already exists
I20240502 14:01:15.801424 26726 test_util.cc:268] Using random seed: 157055820
I20240502 14:01:15.803802 10373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cf72901dd0434d9b88902343d45977" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:15.803957 10373 raft_consensus.cc:3032] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.804000 10373 raft_consensus.cc:729] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.804138 10373 consensus_queue.cc:260] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.804347 10373 raft_consensus.cc:3037] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.805393 10373 raft_consensus.cc:2454] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.807686 9963 catalog_manager.cc:5484] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.808012 9963 catalog_manager.cc:5484] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.809634 10507 leader_election.cc:288] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:15.811187 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.811311 10244 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.812395 10244 raft_consensus.cc:2454] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.812371 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "965596e659e340c7b231e4eef1868e92"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.812482 10507 raft_consensus.cc:482] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.812537 10507 raft_consensus.cc:3037] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.813522 10507 raft_consensus.cc:504] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.814203 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:15.814316 10376 raft_consensus.cc:3032] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.814355 10376 raft_consensus.cc:729] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.814481 10376 consensus_queue.cc:260] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.814643 10376 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.815078 10507 leader_election.cc:288] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:15.815210 10442 leader_election.cc:302] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.815625 12458 raft_consensus.cc:2781] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.815636 10376 raft_consensus.cc:2454] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.815716 12458 raft_consensus.cc:686] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.815872 12458 consensus_queue.cc:237] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.818169 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965596e659e340c7b231e4eef1868e92" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.818302 10113 raft_consensus.cc:3032] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.818344 10113 raft_consensus.cc:729] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:15.818490 10113 consensus_queue.cc:260] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.818475 9963 catalog_manager.cc:5484] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.818672 10113 raft_consensus.cc:3037] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.819456 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9c1b9c544e23474791c93f4094b4c672"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.819561 10507 raft_consensus.cc:482] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.819618 10507 raft_consensus.cc:3037] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.819703 10113 raft_consensus.cc:2454] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.820569 10507 raft_consensus.cc:504] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.820907 10507 leader_election.cc:288] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:15.821043 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965596e659e340c7b231e4eef1868e92" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.821171 10243 raft_consensus.cc:3037] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.821519 10506 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a54336e39ef249fdb7eb6b50cb5b1cda"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.821609 10506 raft_consensus.cc:482] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.821653 10506 raft_consensus.cc:3037] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.822151 10243 raft_consensus.cc:2454] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.822566 10506 raft_consensus.cc:504] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.822957 10506 leader_election.cc:288] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:15.823783 10506 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aeceb15b637841609765e29e9b4d7ef1"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.823911 10506 raft_consensus.cc:482] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.823974 10506 raft_consensus.cc:3037] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.824777 10506 raft_consensus.cc:504] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.825085 10506 leader_election.cc:288] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:15.826138 10507 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0a820d81837c49c3a6a99b09dd5afc49"
dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
from {username='slave'} at 127.0.0.1:51846
I20240502 14:01:15.826238 10507 raft_consensus.cc:482] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.826294 10507 raft_consensus.cc:3037] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.827176 10507 raft_consensus.cc:504] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.827217 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1b9c544e23474791c93f4094b4c672" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.827323 10112 raft_consensus.cc:3037] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.827481 10507 leader_election.cc:288] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:15.827934 10111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54336e39ef249fdb7eb6b50cb5b1cda" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.828039 10111 raft_consensus.cc:3032] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.828040 10376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a820d81837c49c3a6a99b09dd5afc49" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:15.828081 10111 raft_consensus.cc:729] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:15.828142 10376 raft_consensus.cc:3032] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.828182 10376 raft_consensus.cc:729] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.828229 10111 consensus_queue.cc:260] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.828290 10376 consensus_queue.cc:260] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.828382 10111 raft_consensus.cc:3037] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.828446 10376 raft_consensus.cc:3037] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.828397 10112 raft_consensus.cc:2454] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.829293 10376 raft_consensus.cc:2454] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.829291 10111 raft_consensus.cc:2454] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.829524 10110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeceb15b637841609765e29e9b4d7ef1" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.829618 10110 raft_consensus.cc:3037] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.829759 10440 leader_election.cc:302] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.830184 10442 leader_election.cc:302] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.830466 12491 raft_consensus.cc:2781] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.830554 10442 leader_election.cc:302] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.830561 10110 raft_consensus.cc:2454] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.830595 11836 raft_consensus.cc:2781] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.830655 11836 raft_consensus.cc:686] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.830722 10442 leader_election.cc:302] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.830788 11836 consensus_queue.cc:237] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.831359 12417 raft_consensus.cc:2781] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.831382 11836 raft_consensus.cc:2781] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.831450 11836 raft_consensus.cc:686] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.831559 11836 consensus_queue.cc:237] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.832391 10442 leader_election.cc:302] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:15.831449 12417 raft_consensus.cc:686] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.832563 12417 consensus_queue.cc:237] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.833067 12458 raft_consensus.cc:2781] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.833148 12458 raft_consensus.cc:686] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.833214 10242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54336e39ef249fdb7eb6b50cb5b1cda" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.833250 12458 consensus_queue.cc:237] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.833320 10242 raft_consensus.cc:3037] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.834297 10242 raft_consensus.cc:2454] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.834307 10507 raft_consensus.cc:1226] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Rejecting Update request from peer 1ca5103abf454aaca375e9e07eb139cc for earlier term 1. Current term is 2. Ops: [1.5-1.5]
I20240502 14:01:15.834496 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeceb15b637841609765e29e9b4d7ef1" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.834589 10245 raft_consensus.cc:3032] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.834628 10245 raft_consensus.cc:729] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:15.834800 10245 consensus_queue.cc:260] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.834925 10245 raft_consensus.cc:3037] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.835228 26726 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_6 already exists
I20240502 14:01:15.835244 10110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a820d81837c49c3a6a99b09dd5afc49" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.835341 10110 raft_consensus.cc:3037] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:01:15.834945 12499 consensus_queue.cc:1159] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 1ca5103abf454aaca375e9e07eb139cc for earlier term 1. Current term is 2. Ops: [1.5-1.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20240502 14:01:15.835404 26726 test_util.cc:268] Using random seed: 157089802
I20240502 14:01:15.835920 10245 raft_consensus.cc:2396] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 for term 2 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 14:01:15.836337 10110 raft_consensus.cc:2454] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.839855 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1b9c544e23474791c93f4094b4c672" candidate_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.839962 10243 raft_consensus.cc:3032] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.839998 10243 raft_consensus.cc:729] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:15.840116 10243 consensus_queue.cc:260] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.840276 10243 raft_consensus.cc:3037] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.840598 10372 raft_consensus.cc:1261] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.841234 10239 raft_consensus.cc:1261] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.841372 12417 consensus_queue.cc:1035] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.841250 10243 raft_consensus.cc:2454] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5d4bb5f8fa40d5813a5ee8bbfce3c0 in term 2.
I20240502 14:01:15.842311 10113 raft_consensus.cc:1226] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Rejecting Update request from peer 1ca5103abf454aaca375e9e07eb139cc for earlier term 1. Current term is 2. Ops: [1.5-1.5]
I20240502 14:01:15.843163 12417 consensus_queue.cc:1035] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.858790 12491 raft_consensus.cc:686] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Leader. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.858976 12491 consensus_queue.cc:237] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.859375 26726 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_7 already exists
I20240502 14:01:15.859534 26726 test_util.cc:268] Using random seed: 157113931
I20240502 14:01:15.861917 9966 catalog_manager.cc:5484] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:15.862229 9966 catalog_manager.cc:5484] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.862457 9966 catalog_manager.cc:5484] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 068f58913ea3478599df829d5118f7be (127.26.25.129) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.862684 9966 catalog_manager.cc:5484] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 068f58913ea3478599df829d5118f7be (127.26.25.129) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.867522 12399 mvcc.cc:205] Tried to move back new op lower bound from 7023241117038833664 to 7023241116943273984. Current Snapshot: MvccSnapshot[applied={T|T < 7023241117038833664 or (T in {7023241117038833664})}]
I20240502 14:01:15.869100 9966 catalog_manager.cc:5484] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 reported cstate change: term changed from 1 to 2, leader changed from 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130) to ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132). New cstate: current_term: 2 leader_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: HEALTHY } } }
W20240502 14:01:15.881979 10073 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39054: Illegal state: replica 068f58913ea3478599df829d5118f7be is not leader of this config: current role FOLLOWER
I20240502 14:01:15.896944 12403 mvcc.cc:205] Tried to move back new op lower bound from 7023241117038833664 to 7023241116943273984. Current Snapshot: MvccSnapshot[applied={T|T < 7023241117038833664}]
I20240502 14:01:15.904181 10113 raft_consensus.cc:1261] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.904222 10240 raft_consensus.cc:1261] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.904414 26726 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_8 already exists
I20240502 14:01:15.904579 26726 test_util.cc:268] Using random seed: 157158978
W20240502 14:01:15.905067 10202 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51976: Illegal state: replica 1ca5103abf454aaca375e9e07eb139cc is not leader of this config: current role FOLLOWER
I20240502 14:01:15.906003 12453 consensus_queue.cc:1035] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.906235 12510 consensus_queue.cc:1035] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:01:15.912850 10073 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39034: Illegal state: replica 068f58913ea3478599df829d5118f7be is not leader of this config: current role FOLLOWER
I20240502 14:01:15.927001 10243 raft_consensus.cc:1261] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 14:01:15.932018 10243 raft_consensus.cc:1261] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.932859 10114 raft_consensus.cc:1261] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 14:01:15.933326 11836 consensus_queue.cc:1035] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:15.933743 11836 consensus_queue.cc:1035] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.934330 10370 raft_consensus.cc:1261] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.934407 10376 raft_consensus.cc:1261] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.934871 10372 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "35c205ff5aaf46c09fc93b9ded95e971"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:15.935016 10372 raft_consensus.cc:482] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.935079 10372 raft_consensus.cc:3037] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.936199 10372 raft_consensus.cc:504] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.936559 10372 leader_election.cc:288] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:15.937430 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c205ff5aaf46c09fc93b9ded95e971" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.937551 10243 raft_consensus.cc:3037] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.937659 10372 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:15.938060 10114 raft_consensus.cc:1261] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.938552 10243 raft_consensus.cc:2454] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:15.939080 12453 consensus_queue.cc:1035] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.939344 12453 consensus_queue.cc:1035] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.941531 10372 raft_consensus.cc:482] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.941704 12458 consensus_queue.cc:1035] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:15.940829 10311 leader_election.cc:302] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:15.942013 10372 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.942270 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c205ff5aaf46c09fc93b9ded95e971" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:15.942390 10507 raft_consensus.cc:3032] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.942454 10507 raft_consensus.cc:729] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.942166 12452 raft_consensus.cc:2781] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.942608 10507 consensus_queue.cc:260] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.942674 12452 raft_consensus.cc:686] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.942808 12452 consensus_queue.cc:237] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.942891 10507 raft_consensus.cc:3037] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.943617 12453 consensus_queue.cc:1035] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.944001 10507 raft_consensus.cc:2454] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:15.945052 9963 catalog_manager.cc:5484] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 1 to 2, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.945976 10372 raft_consensus.cc:504] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:15.946877 10372 leader_election.cc:288] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:15.949003 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.949120 10114 raft_consensus.cc:3032] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.949159 10114 raft_consensus.cc:729] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:15.949280 10114 consensus_queue.cc:260] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:15.949466 10114 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.950481 10114 raft_consensus.cc:2454] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:15.951376 10311 leader_election.cc:302] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:15.951581 12463 raft_consensus.cc:2781] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.951670 12463 raft_consensus.cc:686] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.951793 12463 consensus_queue.cc:237] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:15.954205 12495 mvcc.cc:205] Tried to move back new op lower bound from 7023241117390934016 to 7023241116710834176. Current Snapshot: MvccSnapshot[applied={T|T < 7023241117390934016 or (T in {7023241117390934016})}]
I20240502 14:01:15.955927 10372 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0577d6cbf5a04722843d7ecf9cf28be9"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:15.956099 10372 raft_consensus.cc:465] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 14:01:15.957475 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.957598 10243 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.958653 10243 raft_consensus.cc:2454] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:15.960305 9963 catalog_manager.cc:5484] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 1 to 2, leader changed from 068f58913ea3478599df829d5118f7be (127.26.25.129) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.960650 10243 raft_consensus.cc:1261] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.961115 10110 raft_consensus.cc:1261] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.963963 12453 consensus_queue.cc:1035] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.966768 10376 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:15.966898 10376 raft_consensus.cc:482] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.966955 10376 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:15.967782 26726 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_9 already exists
I20240502 14:01:15.968058 10376 raft_consensus.cc:504] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.968429 10376 leader_election.cc:288] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:15.969537 10240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.969692 10240 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:15.970901 10240 raft_consensus.cc:2454] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:15.970885 12523 consensus_queue.cc:1035] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.971698 10375 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f0b83b1163014375a53d97d95f4fcb13"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:15.971807 10375 raft_consensus.cc:482] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.971911 10375 raft_consensus.cc:3037] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.972885 10375 raft_consensus.cc:504] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.973117 10311 leader_election.cc:302] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:15.973223 10375 leader_election.cc:288] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:15.973400 12452 raft_consensus.cc:2781] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:15.973484 12452 raft_consensus.cc:686] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.973619 12452 consensus_queue.cc:237] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.974251 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:15.974361 10507 raft_consensus.cc:3032] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:15.974406 10507 raft_consensus.cc:729] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.974449 10241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b83b1163014375a53d97d95f4fcb13" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.974550 10241 raft_consensus.cc:3032] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.974589 10241 raft_consensus.cc:729] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:15.974551 10507 consensus_queue.cc:260] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:15.974681 10507 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:15.974728 10241 consensus_queue.cc:260] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.974937 10241 raft_consensus.cc:3037] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.975719 10507 raft_consensus.cc:2396] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243b53aabd2d4443a8971e707bcc4a51 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 14:01:15.975961 10241 raft_consensus.cc:2454] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:15.976716 10311 leader_election.cc:302] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:15.976889 11821 raft_consensus.cc:2781] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.976989 11821 raft_consensus.cc:686] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.977105 11821 consensus_queue.cc:237] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.977157 10111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b83b1163014375a53d97d95f4fcb13" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.977264 10111 raft_consensus.cc:3037] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.977483 10375 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:15.977578 10375 raft_consensus.cc:482] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.977631 10375 raft_consensus.cc:3037] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.978263 10111 raft_consensus.cc:2454] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:15.978533 10375 raft_consensus.cc:504] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.978859 10375 leader_election.cc:288] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:15.979279 10111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.979380 10111 raft_consensus.cc:3032] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.979434 10111 raft_consensus.cc:729] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:15.979565 10111 consensus_queue.cc:260] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.979660 10241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:15.979748 10241 raft_consensus.cc:3037] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.979725 10111 raft_consensus.cc:3037] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.980252 10375 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2379dbd40b744991be25c2914f878913"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:15.980343 10375 raft_consensus.cc:482] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:15.980392 10375 raft_consensus.cc:3037] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.980669 10241 raft_consensus.cc:2454] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:15.980707 10111 raft_consensus.cc:2454] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:15.980991 10311 leader_election.cc:302] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:15.981545 10375 raft_consensus.cc:504] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.982141 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2379dbd40b744991be25c2914f878913" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:15.982236 10112 raft_consensus.cc:3037] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.982337 12452 raft_consensus.cc:2781] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.982424 12452 raft_consensus.cc:686] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.982530 12452 consensus_queue.cc:237] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:15.983137 10112 raft_consensus.cc:2454] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:15.983637 10311 leader_election.cc:302] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:15.983676 10375 leader_election.cc:288] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:15.983956 11821 raft_consensus.cc:2781] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:15.984035 11821 raft_consensus.cc:686] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:15.984135 11821 consensus_queue.cc:237] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.984200 10112 raft_consensus.cc:1261] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.985602 9971 catalog_manager.cc:5484] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 3 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.987694 12508 consensus_queue.cc:1035] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:15.993005 10373 raft_consensus.cc:1261] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:15.995369 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2379dbd40b744991be25c2914f878913" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:15.995494 10507 raft_consensus.cc:3032] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:15.995535 10507 raft_consensus.cc:729] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:15.995672 10507 consensus_queue.cc:260] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:15.996065 10507 raft_consensus.cc:3037] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:15.997527 10507 raft_consensus.cc:2454] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:15.998312 9971 catalog_manager.cc:5484] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 1 to 2, leader changed from 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:15.998471 12453 consensus_queue.cc:1035] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.000345 10370 raft_consensus.cc:1226] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Rejecting Update request from peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0 for earlier term 1. Current term is 2. Ops: []
I20240502 14:01:15.999876 9971 catalog_manager.cc:5484] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 1 to 2, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.001307 9971 catalog_manager.cc:5484] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 1 to 2, leader changed from 068f58913ea3478599df829d5118f7be (127.26.25.129) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.002843 10375 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.002957 10375 raft_consensus.cc:482] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.003016 10375 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
W20240502 14:01:16.009182 10466 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51894: Illegal state: replica ee5d4bb5f8fa40d5813a5ee8bbfce3c0 is not leader of this config: current role FOLLOWER
I20240502 14:01:16.015920 10375 raft_consensus.cc:504] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.016297 10375 leader_election.cc:288] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.016628 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.016768 10243 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.017028 10372 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bb0d4740d5ca4d80a80751151b3f57a7"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.017132 10372 raft_consensus.cc:482] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.017186 10372 raft_consensus.cc:3037] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.017813 10243 raft_consensus.cc:2454] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.018064 10311 leader_election.cc:302] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.018261 10372 raft_consensus.cc:504] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.018615 10372 leader_election.cc:288] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:16.018867 12452 raft_consensus.cc:2781] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.018962 12452 raft_consensus.cc:686] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.019080 12452 consensus_queue.cc:237] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.022989 9971 catalog_manager.cc:5484] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 3 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.023856 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d4740d5ca4d80a80751151b3f57a7" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.024036 10112 raft_consensus.cc:3037] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.025218 10112 raft_consensus.cc:2454] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:16.025776 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d4740d5ca4d80a80751151b3f57a7" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.025893 10244 raft_consensus.cc:3032] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:16.025949 10244 raft_consensus.cc:729] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.026077 10244 consensus_queue.cc:260] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.026262 10244 raft_consensus.cc:3037] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.027266 10244 raft_consensus.cc:2454] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:16.027900 10376 raft_consensus.cc:1226] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Rejecting Update request from peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0 for earlier term 2. Current term is 3. Ops: [2.12-2.12]
I20240502 14:01:16.028822 10311 leader_election.cc:302] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.029273 11821 raft_consensus.cc:2781] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:16.029346 11821 raft_consensus.cc:686] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.029466 11821 consensus_queue.cc:237] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.029790 10506 raft_consensus.cc:1261] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:01:16.030793 10376 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "45f819d4fb5946248f5ec084d252bcd0"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.030858 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.030887 10376 raft_consensus.cc:482] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.030948 10376 raft_consensus.cc:3037] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.030967 10507 raft_consensus.cc:3032] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.031008 10507 raft_consensus.cc:729] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.031240 10507 consensus_queue.cc:260] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.12, Last appended by leader: 12, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.031345 10507 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.031632 10112 raft_consensus.cc:1261] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:01:16.031946 10376 raft_consensus.cc:504] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.032302 10376 leader_election.cc:288] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.032395 10507 raft_consensus.cc:2396] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243b53aabd2d4443a8971e707bcc4a51 for term 3 because replica has last-logged OpId of term: 2 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 14:01:16.033320 12452 consensus_queue.cc:1035] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:16.033850 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f819d4fb5946248f5ec084d252bcd0" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.033975 10114 raft_consensus.cc:3037] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.033919 9966 catalog_manager.cc:5484] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 1 to 2, leader changed from 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.035089 10114 raft_consensus.cc:2454] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.036065 10374 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c958a9027a064dcb9026608541a22883"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.036178 10374 raft_consensus.cc:482] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.036245 10374 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.037009 12463 consensus_queue.cc:1035] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:16.037245 10374 raft_consensus.cc:504] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.037400 10311 leader_election.cc:302] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.037567 10374 leader_election.cc:288] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.037567 12463 raft_consensus.cc:2781] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.037678 12463 raft_consensus.cc:686] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.037794 12463 consensus_queue.cc:237] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.040704 10244 raft_consensus.cc:1226] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Rejecting Update request from peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0 for earlier term 2. Current term is 3. Ops: [2.12-2.12]
I20240502 14:01:16.040745 10374 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.040854 10374 raft_consensus.cc:482] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.040906 10374 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.041155 9963 catalog_manager.cc:5484] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 3 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.041574 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c958a9027a064dcb9026608541a22883" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.041678 10112 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.041823 10374 raft_consensus.cc:504] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.042150 10374 leader_election.cc:288] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:16.042950 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.043062 10245 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.044203 10245 raft_consensus.cc:2454] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.045457 10374 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a804b61491304055865347e70e8a9778"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.045580 10374 raft_consensus.cc:482] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.045583 10112 raft_consensus.cc:2454] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.045641 10374 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.046653 10374 raft_consensus.cc:504] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.046849 10375 raft_consensus.cc:1226] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Rejecting Update request from peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0 for earlier term 2. Current term is 3. Ops: []
I20240502 14:01:16.046983 10374 leader_election.cc:288] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:16.047003 10311 leader_election.cc:302] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.047562 12463 raft_consensus.cc:2781] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.047641 10374 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "657170da057f4318a6165d2ed9097e58"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.047646 12463 raft_consensus.cc:686] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.047801 10374 raft_consensus.cc:482] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.047895 10374 raft_consensus.cc:3037] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.047859 12463 consensus_queue.cc:237] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.048432 12417 consensus_queue.cc:1046] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: INVALID_TERM, Last received: 2.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:16.048779 11836 raft_consensus.cc:3032] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.048832 11836 raft_consensus.cc:729] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.048933 10374 raft_consensus.cc:504] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.048943 11836 consensus_queue.cc:260] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.049268 10374 leader_election.cc:288] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:16.049374 11836 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.050405 10374 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9a76fe53d2dd4b12bc04ac52089a7120"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.050519 10374 raft_consensus.cc:482] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.050578 10374 raft_consensus.cc:3037] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.051067 10311 leader_election.cc:302] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.052579 12463 raft_consensus.cc:2781] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.052659 12463 raft_consensus.cc:686] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.052698 10374 raft_consensus.cc:504] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.052760 12463 consensus_queue.cc:237] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.053040 10374 leader_election.cc:288] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.053782 10374 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "67cf72901dd0434d9b88902343d45977"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.053750 9971 catalog_manager.cc:5484] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 3 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.053889 10374 raft_consensus.cc:482] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.053951 10374 raft_consensus.cc:3037] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.054698 10374 raft_consensus.cc:504] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.055104 10240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a804b61491304055865347e70e8a9778" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.055272 10240 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.055604 10374 leader_election.cc:288] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:16.056309 10374 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.056417 10374 raft_consensus.cc:482] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.056480 10374 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.056578 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c958a9027a064dcb9026608541a22883" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.056690 10506 raft_consensus.cc:3032] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.056731 10506 raft_consensus.cc:729] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.056823 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657170da057f4318a6165d2ed9097e58" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.056871 10506 consensus_queue.cc:260] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.056949 10244 raft_consensus.cc:3032] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:16.057000 10244 raft_consensus.cc:729] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.057027 10506 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.057169 10244 consensus_queue.cc:260] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.057340 10244 raft_consensus.cc:3037] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.058036 10506 raft_consensus.cc:2454] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.058063 10376 raft_consensus.cc:1226] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Rejecting Update request from peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0 for earlier term 2. Current term is 3. Ops: []
I20240502 14:01:16.058312 10244 raft_consensus.cc:2454] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:16.058846 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.058950 10506 raft_consensus.cc:3032] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.059001 10506 raft_consensus.cc:729] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.059129 10506 consensus_queue.cc:260] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.15, Last appended by leader: 15, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.059264 10506 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.059513 10311 leader_election.cc:302] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.059790 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cf72901dd0434d9b88902343d45977" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.059945 10113 raft_consensus.cc:3037] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.060284 11821 raft_consensus.cc:2781] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:16.060361 11821 raft_consensus.cc:686] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.060473 11821 consensus_queue.cc:237] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.061007 10113 raft_consensus.cc:2454] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.061276 10374 raft_consensus.cc:504] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.061573 10374 leader_election.cc:288] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:16.061980 10240 raft_consensus.cc:2454] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.062114 10506 raft_consensus.cc:2396] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243b53aabd2d4443a8971e707bcc4a51 for term 3 because replica has last-logged OpId of term: 2 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 14:01:16.062547 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a804b61491304055865347e70e8a9778" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.062610 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a76fe53d2dd4b12bc04ac52089a7120" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.062713 10508 raft_consensus.cc:3032] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.062762 10508 raft_consensus.cc:729] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.062897 10508 consensus_queue.cc:260] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.7, Last appended by leader: 7, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.063040 10508 raft_consensus.cc:3037] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.062744 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657170da057f4318a6165d2ed9097e58" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.063309 10112 raft_consensus.cc:3037] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.063380 10506 raft_consensus.cc:2454] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.063581 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.063679 10244 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.064085 10508 raft_consensus.cc:2396] T 9a76fe53d2dd4b12bc04ac52089a7120 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243b53aabd2d4443a8971e707bcc4a51 for term 3 because replica has last-logged OpId of term: 2 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 6.
I20240502 14:01:16.064240 10311 leader_election.cc:302] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.064364 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cf72901dd0434d9b88902343d45977" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.064421 11821 raft_consensus.cc:2781] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.064468 10508 raft_consensus.cc:3032] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.064491 11821 raft_consensus.cc:686] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.064508 10508 raft_consensus.cc:729] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.064555 10244 raft_consensus.cc:2454] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.064610 10508 consensus_queue.cc:260] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.064606 11821 consensus_queue.cc:237] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.10, Last appended by leader: 10, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.064718 10508 raft_consensus.cc:3037] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.064878 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a76fe53d2dd4b12bc04ac52089a7120" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.064954 10114 raft_consensus.cc:3037] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.065591 10508 raft_consensus.cc:2396] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243b53aabd2d4443a8971e707bcc4a51 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 14:01:16.065740 10114 raft_consensus.cc:2454] T 9a76fe53d2dd4b12bc04ac52089a7120 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.065852 10311 leader_election.cc:302] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.065938 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.066040 10508 raft_consensus.cc:3032] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.066080 10508 raft_consensus.cc:729] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.066200 10508 consensus_queue.cc:260] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.066322 10508 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.066433 12463 raft_consensus.cc:2781] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.066509 12463 raft_consensus.cc:686] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.066613 12463 consensus_queue.cc:237] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.067065 10508 raft_consensus.cc:2454] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.068106 10112 raft_consensus.cc:2454] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:16.067425 10311 leader_election.cc:302] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.068634 11821 raft_consensus.cc:2781] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.068708 11821 raft_consensus.cc:686] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.068806 11821 consensus_queue.cc:237] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.070730 10240 raft_consensus.cc:1261] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:16.071233 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f819d4fb5946248f5ec084d252bcd0" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.071331 10507 raft_consensus.cc:3032] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.071375 10507 raft_consensus.cc:729] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.071453 10111 raft_consensus.cc:1261] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:16.071543 10507 consensus_queue.cc:260] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.071696 10507 raft_consensus.cc:3037] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.072315 10311 leader_election.cc:302] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters: ee5d4bb5f8fa40d5813a5ee8bbfce3c0
I20240502 14:01:16.072495 10507 raft_consensus.cc:2454] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.072634 12463 raft_consensus.cc:2781] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.072714 12463 raft_consensus.cc:686] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.072825 12463 consensus_queue.cc:237] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.075930 10375 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b0f4a67e31b04fefac38cf7d7cb983e7"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.076041 10375 raft_consensus.cc:482] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.076086 10375 raft_consensus.cc:3037] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.080900 11836 consensus_queue.cc:1035] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.081378 12491 consensus_queue.cc:1035] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:01:16.090411 10465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51888: Illegal state: replica ee5d4bb5f8fa40d5813a5ee8bbfce3c0 is not leader of this config: current role FOLLOWER
W20240502 14:01:16.091740 10465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51880: Illegal state: replica ee5d4bb5f8fa40d5813a5ee8bbfce3c0 is not leader of this config: current role FOLLOWER
W20240502 14:01:16.095163 10073 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39034: Illegal state: replica 068f58913ea3478599df829d5118f7be is not leader of this config: current role FOLLOWER
W20240502 14:01:16.097486 10465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51894: Illegal state: replica ee5d4bb5f8fa40d5813a5ee8bbfce3c0 is not leader of this config: current role FOLLOWER
I20240502 14:01:16.104616 10244 raft_consensus.cc:1261] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 14:01:16.104970 11979 heartbeater.cc:503] Master 127.26.25.190:39653 was elected leader, sending a full tablet report...
I20240502 14:01:16.106758 10507 raft_consensus.cc:1261] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 14:01:16.107721 10375 raft_consensus.cc:504] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.108141 10375 leader_election.cc:288] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:16.109645 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f4a67e31b04fefac38cf7d7cb983e7" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.109755 10244 raft_consensus.cc:3037] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.110766 10244 raft_consensus.cc:2454] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:16.111235 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f4a67e31b04fefac38cf7d7cb983e7" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.111340 10112 raft_consensus.cc:3032] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:16.111377 10112 raft_consensus.cc:729] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:16.111534 10112 consensus_queue.cc:260] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.111716 10112 raft_consensus.cc:3037] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.112742 10112 raft_consensus.cc:2454] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:16.114500 10370 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.114638 10370 raft_consensus.cc:482] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.114704 10370 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.115744 10370 raft_consensus.cc:504] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.116098 10370 leader_election.cc:288] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.116456 11821 consensus_queue.cc:1035] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.116559 11821 consensus_queue.cc:1225] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 14:01:16.116607 12463 consensus_queue.cc:1035] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.116806 10375 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e5936fe6a561440281de30e976d12fb3"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.117053 10375 raft_consensus.cc:482] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.117115 10375 raft_consensus.cc:3037] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.120131 10311 leader_election.cc:302] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.120292 11821 raft_consensus.cc:2781] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:16.120416 11821 raft_consensus.cc:686] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.120544 11821 consensus_queue.cc:237] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.121526 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.121659 10244 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.122828 10244 raft_consensus.cc:2454] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.125502 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.125631 10507 raft_consensus.cc:3032] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.125684 10507 raft_consensus.cc:729] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.125819 10507 consensus_queue.cc:260] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.125994 10507 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.126890 10508 pending_rounds.cc:76] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Aborting all ops after (but not including) 3
I20240502 14:01:16.126964 10508 pending_rounds.cc:97] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 14:01:16.127017 10507 raft_consensus.cc:2454] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 3.
I20240502 14:01:16.127041 10508 raft_consensus.cc:2866] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 14:01:16.133316 10375 raft_consensus.cc:504] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.133700 10375 leader_election.cc:288] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.135797 10372 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8b1dff0b06824632869c970e3a031d3b"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.135977 10372 raft_consensus.cc:482] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.136040 10372 raft_consensus.cc:3037] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.137169 10372 raft_consensus.cc:504] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.137501 10372 leader_election.cc:288] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.138309 10372 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fb32f5457e4345a68b283be8891ce202"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.138414 10372 raft_consensus.cc:482] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.138470 10372 raft_consensus.cc:3037] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.139281 10372 raft_consensus.cc:504] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.139566 10372 leader_election.cc:288] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:16.140249 10374 raft_consensus.cc:1226] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Rejecting Update request from peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0 for earlier term 1. Current term is 2. Ops: [1.4-1.4]
I20240502 14:01:16.140471 10372 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0a820d81837c49c3a6a99b09dd5afc49"
dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
from {username='slave'} at 127.0.0.1:54342
I20240502 14:01:16.140575 10372 raft_consensus.cc:482] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.140631 10372 raft_consensus.cc:3037] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.140805 12508 consensus_queue.cc:1046] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.140996 12458 raft_consensus.cc:3032] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:16.141057 12458 raft_consensus.cc:729] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.141191 12458 consensus_queue.cc:260] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.141371 12458 raft_consensus.cc:3037] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.141429 10372 raft_consensus.cc:504] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.141737 10372 leader_election.cc:288] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.144089 10372 raft_consensus.cc:1226] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Rejecting Update request from peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0 for earlier term 2. Current term is 3. Ops: [2.8-2.8]
I20240502 14:01:16.144457 12523 consensus_queue.cc:1046] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: INVALID_TERM, Last received: 2.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:16.144685 12458 raft_consensus.cc:3032] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.144742 12458 raft_consensus.cc:729] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.144870 12458 consensus_queue.cc:260] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.144805 9971 catalog_manager.cc:5484] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 3 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.145002 12458 raft_consensus.cc:3037] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.145080 9971 catalog_manager.cc:5484] T 9a76fe53d2dd4b12bc04ac52089a7120 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 3 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.151506 10245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5936fe6a561440281de30e976d12fb3" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.151646 10245 raft_consensus.cc:3037] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.152555 10245 raft_consensus.cc:2396] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 243b53aabd2d4443a8971e707bcc4a51 for term 2 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 14:01:16.153316 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5936fe6a561440281de30e976d12fb3" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.153472 10508 raft_consensus.cc:2396] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 243b53aabd2d4443a8971e707bcc4a51 for term 2 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 14:01:16.153666 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b1dff0b06824632869c970e3a031d3b" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.153786 10113 raft_consensus.cc:3037] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.154686 10112 raft_consensus.cc:1261] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:16.154976 10113 raft_consensus.cc:2454] T 8b1dff0b06824632869c970e3a031d3b P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:16.155462 10245 raft_consensus.cc:1261] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 14:01:16.157668 10245 raft_consensus.cc:1261] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240502 14:01:16.159973 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb32f5457e4345a68b283be8891ce202" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.160136 10113 raft_consensus.cc:3032] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:16.160199 10113 raft_consensus.cc:729] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:16.160486 10113 consensus_queue.cc:260] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.160760 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b1dff0b06824632869c970e3a031d3b" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.160880 10507 raft_consensus.cc:3032] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:16.160934 10507 raft_consensus.cc:729] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.161085 10507 consensus_queue.cc:260] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.161262 10507 raft_consensus.cc:3037] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.162351 10507 raft_consensus.cc:2454] T 8b1dff0b06824632869c970e3a031d3b P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:16.162988 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a820d81837c49c3a6a99b09dd5afc49" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.163137 10507 raft_consensus.cc:2396] T 0a820d81837c49c3a6a99b09dd5afc49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243b53aabd2d4443a8971e707bcc4a51 for term 3 because replica has last-logged OpId of term: 2 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 7.
I20240502 14:01:16.163364 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.163419 10113 raft_consensus.cc:3037] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.163508 10245 raft_consensus.cc:482] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.163581 10245 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.164535 10113 raft_consensus.cc:2454] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:16.164538 10245 raft_consensus.cc:504] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:16.165022 10244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb32f5457e4345a68b283be8891ce202" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.165123 10245 leader_election.cc:288] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:16.165163 10244 raft_consensus.cc:3037] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.165737 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "35c205ff5aaf46c09fc93b9ded95e971"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.165861 10245 raft_consensus.cc:482] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.165930 10245 raft_consensus.cc:3037] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.166276 10244 raft_consensus.cc:2454] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243b53aabd2d4443a8971e707bcc4a51 in term 2.
I20240502 14:01:16.166743 10245 raft_consensus.cc:504] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.167141 10245 leader_election.cc:288] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.167382 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c205ff5aaf46c09fc93b9ded95e971" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.167491 10507 raft_consensus.cc:3037] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.167758 10508 raft_consensus.cc:1261] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240502 14:01:16.168013 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.168112 10113 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.168460 10507 raft_consensus.cc:2454] T 35c205ff5aaf46c09fc93b9ded95e971 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.168844 10180 leader_election.cc:302] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:16.169034 12117 raft_consensus.cc:2781] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.169111 12117 raft_consensus.cc:686] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.169247 12117 consensus_queue.cc:237] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.170425 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0577d6cbf5a04722843d7ecf9cf28be9"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.170529 10245 raft_consensus.cc:482] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.170580 10245 raft_consensus.cc:3037] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.171329 9966 catalog_manager.cc:5484] T 35c205ff5aaf46c09fc93b9ded95e971 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.172011 10245 raft_consensus.cc:504] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.172320 10245 leader_election.cc:288] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:16.172430 10113 raft_consensus.cc:2454] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.172747 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a820d81837c49c3a6a99b09dd5afc49" candidate_uuid: "243b53aabd2d4443a8971e707bcc4a51" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.172811 10180 leader_election.cc:302] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.172852 10112 raft_consensus.cc:3037] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.174046 10112 raft_consensus.cc:2396] T 0a820d81837c49c3a6a99b09dd5afc49 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 243b53aabd2d4443a8971e707bcc4a51 for term 3 because replica has last-logged OpId of term: 2 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 7.
I20240502 14:01:16.174268 12117 raft_consensus.cc:2781] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.174350 12117 raft_consensus.cc:686] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.174371 11821 consensus_queue.cc:1035] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.174472 12117 consensus_queue.cc:237] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:16.174871 10311 leader_election.cc:302] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.175031 11821 raft_consensus.cc:2781] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:16.175158 11821 raft_consensus.cc:686] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.175287 11821 consensus_queue.cc:237] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.176223 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.176379 10245 raft_consensus.cc:482] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.176494 10245 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.176877 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0577d6cbf5a04722843d7ecf9cf28be9" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.176995 10112 raft_consensus.cc:3037] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.177517 9966 catalog_manager.cc:5484] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.177955 10311 leader_election.cc:302] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.178118 9971 catalog_manager.cc:5484] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 3 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.178350 9971 catalog_manager.cc:5484] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 3 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.178596 9971 catalog_manager.cc:5484] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 3 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.178787 9971 catalog_manager.cc:5484] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 1 to 2, leader changed from 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.179003 11821 raft_consensus.cc:2781] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:16.179147 11821 raft_consensus.cc:686] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.179270 11821 consensus_queue.cc:237] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.180199 10311 leader_election.cc:302] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:16.180414 12463 raft_consensus.cc:2781] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.180516 12463 raft_consensus.cc:686] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Leader. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.180644 12463 consensus_queue.cc:237] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.181799 10245 raft_consensus.cc:504] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.182523 10311 leader_election.cc:302] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51; no voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0
I20240502 14:01:16.182760 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.182821 10112 raft_consensus.cc:2454] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 2.
I20240502 14:01:16.182880 10507 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.183080 10180 leader_election.cc:302] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.183226 12499 raft_consensus.cc:2781] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:16.183284 12499 raft_consensus.cc:686] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 2 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.183388 12499 consensus_queue.cc:237] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.183951 10507 raft_consensus.cc:2454] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 4.
I20240502 14:01:16.184341 10245 leader_election.cc:288] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.184796 10180 leader_election.cc:302] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:16.185089 12117 raft_consensus.cc:2781] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Leader election won for term 4
I20240502 14:01:16.185168 12117 raft_consensus.cc:686] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [term 4 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.185292 12117 consensus_queue.cc:237] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.185286 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e5936fe6a561440281de30e976d12fb3"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.185954 10245 raft_consensus.cc:482] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.186067 10245 raft_consensus.cc:3037] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.186086 9966 catalog_manager.cc:5484] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 1 to 2, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 2 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.187186 12452 raft_consensus.cc:2726] T 0a820d81837c49c3a6a99b09dd5afc49 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 14:01:16.187275 12452 consensus_queue.cc:1035] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:16.188274 10311 leader_election.cc:302] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 243b53aabd2d4443a8971e707bcc4a51; no voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0
I20240502 14:01:16.188400 12452 raft_consensus.cc:2726] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20240502 14:01:16.188606 12452 consensus_queue.cc:1035] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.189059 12452 consensus_queue.cc:1035] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:16.189882 10112 raft_consensus.cc:1226] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Rejecting Update request from peer 243b53aabd2d4443a8971e707bcc4a51 for earlier term 1. Current term is 2. Ops: []
I20240502 14:01:16.191219 10112 raft_consensus.cc:1226] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Rejecting Update request from peer 243b53aabd2d4443a8971e707bcc4a51 for earlier term 2. Current term is 3. Ops: [2.4-2.5]
I20240502 14:01:16.191597 9966 catalog_manager.cc:5484] T bb4efdb1ee0d43558cdc3106829b159c P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 3 to 4, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 4 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } }
W20240502 14:01:16.192867 10073 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39052: Illegal state: replica 068f58913ea3478599df829d5118f7be is not leader of this config: current role FOLLOWER
I20240502 14:01:16.193904 10507 raft_consensus.cc:1226] T bb4efdb1ee0d43558cdc3106829b159c P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 4 FOLLOWER]: Rejecting Update request from peer 243b53aabd2d4443a8971e707bcc4a51 for earlier term 3. Current term is 4. Ops: []
I20240502 14:01:16.194839 10245 raft_consensus.cc:504] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.197571 9963 catalog_manager.cc:5484] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 1 to 2, leader changed from 068f58913ea3478599df829d5118f7be (127.26.25.129) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.197826 9963 catalog_manager.cc:5484] T 8b1dff0b06824632869c970e3a031d3b P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 1 to 2, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.198009 9963 catalog_manager.cc:5484] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 3 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.198192 9963 catalog_manager.cc:5484] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 reported cstate change: term changed from 1 to 2, leader changed from 068f58913ea3478599df829d5118f7be (127.26.25.129) to 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131). New cstate: current_term: 2 leader_uuid: "243b53aabd2d4443a8971e707bcc4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.199797 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.199934 10375 raft_consensus.cc:3032] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.199977 10375 raft_consensus.cc:729] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.200122 10375 consensus_queue.cc:260] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:16.200230 10375 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.201933 10375 raft_consensus.cc:2396] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 14:01:16.202354 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0577d6cbf5a04722843d7ecf9cf28be9" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 2 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.202472 10374 raft_consensus.cc:3032] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:16.202513 10374 raft_consensus.cc:729] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.202661 10374 consensus_queue.cc:260] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.202708 10508 raft_consensus.cc:1261] T 45f819d4fb5946248f5ec084d252bcd0 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20240502 14:01:16.202775 10374 raft_consensus.cc:3037] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.203033 10375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c205ff5aaf46c09fc93b9ded95e971" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.203112 10375 raft_consensus.cc:3032] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.203145 10375 raft_consensus.cc:729] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.203295 10375 consensus_queue.cc:260] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.203416 10375 raft_consensus.cc:3037] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.203748 10374 raft_consensus.cc:2396] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 2 because replica has last-logged OpId of term: 1 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 14:01:16.203917 10372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb4efdb1ee0d43558cdc3106829b159c" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.203994 10372 raft_consensus.cc:3032] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 14:01:16.204026 10372 raft_consensus.cc:729] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.204159 10372 consensus_queue.cc:260] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.204288 10372 raft_consensus.cc:3037] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.204358 10375 raft_consensus.cc:2396] T 35c205ff5aaf46c09fc93b9ded95e971 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 14:01:16.205150 10372 raft_consensus.cc:2454] T bb4efdb1ee0d43558cdc3106829b159c P 243b53aabd2d4443a8971e707bcc4a51 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 4.
I20240502 14:01:16.205318 10112 raft_consensus.cc:1261] T 45f819d4fb5946248f5ec084d252bcd0 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20240502 14:01:16.195740 10244 raft_consensus.cc:1226] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 2 LEADER]: Rejecting Update request from peer 243b53aabd2d4443a8971e707bcc4a51 for earlier term 1. Current term is 2. Ops: [1.8-1.8]
I20240502 14:01:16.196174 10239 raft_consensus.cc:1226] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Rejecting Update request from peer 243b53aabd2d4443a8971e707bcc4a51 for earlier term 2. Current term is 3. Ops: [2.4-2.5]
I20240502 14:01:16.207104 10245 leader_election.cc:288] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.207362 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5936fe6a561440281de30e976d12fb3" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.207494 10508 raft_consensus.cc:3037] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.208101 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fb32f5457e4345a68b283be8891ce202"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.209234 12463 consensus_queue.cc:1035] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:16.209295 10245 raft_consensus.cc:482] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.209530 12463 consensus_queue.cc:1035] T 45f819d4fb5946248f5ec084d252bcd0 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:16.209815 10245 raft_consensus.cc:3037] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.212433 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5936fe6a561440281de30e976d12fb3" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.212539 10374 raft_consensus.cc:3037] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
W20240502 14:01:16.212601 10073 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39062: Illegal state: replica 068f58913ea3478599df829d5118f7be is not leader of this config: current role FOLLOWER
I20240502 14:01:16.213497 10374 raft_consensus.cc:2454] T e5936fe6a561440281de30e976d12fb3 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.213713 10508 raft_consensus.cc:2454] T e5936fe6a561440281de30e976d12fb3 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.214051 10180 leader_election.cc:302] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:16.214249 12117 raft_consensus.cc:2781] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.214341 12117 raft_consensus.cc:686] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.214529 10245 raft_consensus.cc:504] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.215044 10239 raft_consensus.cc:1226] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Rejecting Update request from peer 243b53aabd2d4443a8971e707bcc4a51 for earlier term 2. Current term is 3. Ops: []
I20240502 14:01:16.215055 10245 leader_election.cc:288] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:16.214504 12117 consensus_queue.cc:237] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.216562 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.216674 10245 raft_consensus.cc:482] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.216732 10245 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.217464 10239 raft_consensus.cc:1261] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 5. (term mismatch)
I20240502 14:01:16.217527 10239 pending_rounds.cc:76] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc: Aborting all ops after (but not including) 4
I20240502 14:01:16.217559 10239 pending_rounds.cc:97] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc: Aborting uncommitted WRITE_OP operation due to leader change: 1.5
W20240502 14:01:16.217701 10239 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51980: Aborted: Op aborted by new leader
I20240502 14:01:16.220258 12557 consensus_queue.cc:1035] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:16.220989 10114 raft_consensus.cc:1261] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:01:16.224236 9971 catalog_manager.cc:5484] T e5936fe6a561440281de30e976d12fb3 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 1 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:16.229105 10073 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39072: Illegal state: replica 068f58913ea3478599df829d5118f7be is not leader of this config: current role FOLLOWER
W20240502 14:01:16.229477 10202 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:52010: Illegal state: replica 1ca5103abf454aaca375e9e07eb139cc is not leader of this config: current role FOLLOWER
I20240502 14:01:16.229851 12586 consensus_queue.cc:1035] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:16.230108 12463 consensus_queue.cc:1046] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:16.230314 11821 raft_consensus.cc:3032] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.230422 11821 raft_consensus.cc:729] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.230705 11821 consensus_queue.cc:260] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.9, Last appended by leader: 9, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.231341 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb32f5457e4345a68b283be8891ce202" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.235034 12534 mvcc.cc:205] Tried to move back new op lower bound from 7023241118511448064 to 7023241117852139520. Current Snapshot: MvccSnapshot[applied={T|T < 7023241118511448064}]
I20240502 14:01:16.240842 11821 raft_consensus.cc:3037] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.242174 10374 raft_consensus.cc:2396] T fb32f5457e4345a68b283be8891ce202 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 3 because replica has last-logged OpId of term: 2 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 14:01:16.242832 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb32f5457e4345a68b283be8891ce202" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.242966 10112 raft_consensus.cc:3037] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.243192 10113 raft_consensus.cc:1261] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:01:16.243605 12557 consensus_queue.cc:1035] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:16.244086 10112 raft_consensus.cc:2396] T fb32f5457e4345a68b283be8891ce202 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 3 because replica has last-logged OpId of term: 2 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 14:01:16.245069 10245 raft_consensus.cc:504] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.245669 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.245779 10114 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.246762 10114 raft_consensus.cc:2454] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.247062 10180 leader_election.cc:302] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.247238 12117 raft_consensus.cc:2781] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.247318 12117 raft_consensus.cc:686] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.247460 12117 consensus_queue.cc:237] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.248576 10245 leader_election.cc:288] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.249850 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.249970 10508 raft_consensus.cc:3032] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.250012 10508 raft_consensus.cc:729] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.250167 10508 consensus_queue.cc:260] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.250325 10508 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.251345 10114 raft_consensus.cc:1261] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240502 14:01:16.251426 10178 leader_election.cc:302] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc; no voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51
I20240502 14:01:16.251575 12499 raft_consensus.cc:2726] T fb32f5457e4345a68b283be8891ce202 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 14:01:16.252082 10239 raft_consensus.cc:1261] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240502 14:01:16.252378 10508 raft_consensus.cc:2454] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.253924 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.254031 10245 raft_consensus.cc:482] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.254079 10245 raft_consensus.cc:3037] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.258383 9963 catalog_manager.cc:5484] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.261875 10245 raft_consensus.cc:504] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.262514 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.262640 10114 raft_consensus.cc:3037] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.263674 10114 raft_consensus.cc:2454] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.264037 10180 leader_election.cc:302] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.264214 12499 raft_consensus.cc:2781] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.264305 12499 raft_consensus.cc:686] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.264423 12499 consensus_queue.cc:237] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.265713 10245 leader_election.cc:288] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:16.266386 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.266505 10245 raft_consensus.cc:482] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.266572 10245 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.267621 10245 raft_consensus.cc:504] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.268409 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.268527 10508 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.268828 9965 catalog_manager.cc:5484] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.269542 10508 raft_consensus.cc:2396] T 0b80a9c354354433a48a0a5f8938265d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 4 because replica has last-logged OpId of term: 2 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 14:01:16.269897 10245 leader_election.cc:288] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.270717 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bb0d4740d5ca4d80a80751151b3f57a7"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.270821 10245 raft_consensus.cc:482] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.270880 10245 raft_consensus.cc:3037] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.271901 10112 raft_consensus.cc:1261] T c211d47d28ae45ed9bdf0471b553edb1 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 14:01:16.272015 10245 raft_consensus.cc:504] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.272454 12458 consensus_queue.cc:1035] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.272593 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d4740d5ca4d80a80751151b3f57a7" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.272686 10114 raft_consensus.cc:3037] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.273653 10114 raft_consensus.cc:2454] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.273957 10180 leader_election.cc:302] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.274261 12117 raft_consensus.cc:2781] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.274353 12117 raft_consensus.cc:686] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.274303 12587 consensus_queue.cc:1035] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:16.274477 12117 consensus_queue.cc:237] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.275763 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.275900 10374 raft_consensus.cc:3032] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.275945 10374 raft_consensus.cc:729] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.276078 10374 consensus_queue.cc:260] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.9, Last appended by leader: 9, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.276211 10374 raft_consensus.cc:3037] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.276222 10508 raft_consensus.cc:1261] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 14:01:16.276613 10114 raft_consensus.cc:1261] T 67cf72901dd0434d9b88902343d45977 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Refusing update from remote peer 243b53aabd2d4443a8971e707bcc4a51: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 14:01:16.277226 10374 raft_consensus.cc:2396] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 3 because replica has last-logged OpId of term: 2 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 8.
I20240502 14:01:16.277969 10245 leader_election.cc:288] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:16.278625 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a804b61491304055865347e70e8a9778"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.278744 10245 raft_consensus.cc:482] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.278805 10245 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.278683 9965 catalog_manager.cc:5484] T bb0d4740d5ca4d80a80751151b3f57a7 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.279853 10245 raft_consensus.cc:504] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.280539 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a804b61491304055865347e70e8a9778" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.280665 10508 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.281687 10508 raft_consensus.cc:2454] T a804b61491304055865347e70e8a9778 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 4.
I20240502 14:01:16.282080 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b80a9c354354433a48a0a5f8938265d" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.282155 10180 leader_election.cc:302] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:16.282189 10374 raft_consensus.cc:3032] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 14:01:16.282227 10374 raft_consensus.cc:729] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.282318 12499 raft_consensus.cc:2781] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Leader election won for term 4
I20240502 14:01:16.282382 12499 raft_consensus.cc:686] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [term 4 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.282362 10374 consensus_queue.cc:260] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.282512 10374 raft_consensus.cc:3037] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.282487 12499 consensus_queue.cc:237] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.283769 10245 leader_election.cc:288] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.284456 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "657170da057f4318a6165d2ed9097e58"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.284559 10245 raft_consensus.cc:482] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.284607 10245 raft_consensus.cc:3037] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.285567 10245 raft_consensus.cc:504] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.286240 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657170da057f4318a6165d2ed9097e58" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.286373 10114 raft_consensus.cc:3037] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.286954 10374 raft_consensus.cc:2396] T 0b80a9c354354433a48a0a5f8938265d P 243b53aabd2d4443a8971e707bcc4a51 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 4 because replica has last-logged OpId of term: 3 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 14:01:16.287375 10114 raft_consensus.cc:2454] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.287693 10180 leader_election.cc:302] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.287917 12117 raft_consensus.cc:2781] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.288003 12117 raft_consensus.cc:686] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.287899 9965 catalog_manager.cc:5484] T a804b61491304055865347e70e8a9778 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 3 to 4, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 4 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.288129 12117 consensus_queue.cc:237] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.288825 10245 leader_election.cc:288] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:16.289748 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c211d47d28ae45ed9bdf0471b553edb1"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.289851 10245 raft_consensus.cc:482] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.289902 10245 raft_consensus.cc:3037] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.290813 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d4740d5ca4d80a80751151b3f57a7" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.290915 10374 raft_consensus.cc:3032] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.290951 10374 raft_consensus.cc:729] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.291432 10374 consensus_queue.cc:260] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.9, Last appended by leader: 9, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.291471 10245 raft_consensus.cc:504] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:16.291676 10374 raft_consensus.cc:3037] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.292102 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c211d47d28ae45ed9bdf0471b553edb1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.292229 10508 raft_consensus.cc:3032] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.292279 10508 raft_consensus.cc:729] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.292434 10508 consensus_queue.cc:260] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:16.292578 10508 raft_consensus.cc:3037] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.293025 10374 raft_consensus.cc:2396] T bb0d4740d5ca4d80a80751151b3f57a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 3 because replica has last-logged OpId of term: 2 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 8.
I20240502 14:01:16.293213 12110 heartbeater.cc:503] Master 127.26.25.190:39653 was elected leader, sending a full tablet report...
I20240502 14:01:16.293605 10508 raft_consensus.cc:2396] T c211d47d28ae45ed9bdf0471b553edb1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 14:01:16.293921 9965 catalog_manager.cc:5484] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.294380 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c211d47d28ae45ed9bdf0471b553edb1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.294816 10180 leader_election.cc:302] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc; no voters: 068f58913ea3478599df829d5118f7be, ee5d4bb5f8fa40d5813a5ee8bbfce3c0
I20240502 14:01:16.294988 12499 raft_consensus.cc:2726] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 14:01:16.295059 10245 leader_election.cc:288] T c211d47d28ae45ed9bdf0471b553edb1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:16.295782 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.295965 10245 raft_consensus.cc:482] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.296025 10245 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.296837 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a804b61491304055865347e70e8a9778" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.297037 10374 raft_consensus.cc:3032] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 14:01:16.297078 10245 raft_consensus.cc:504] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.297124 10374 raft_consensus.cc:729] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.297353 10374 consensus_queue.cc:260] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.11, Last appended by leader: 11, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.297708 10374 raft_consensus.cc:3037] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.297729 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.297838 10508 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.298693 10508 raft_consensus.cc:2454] T 10bcbc135aa64d74bd5aaef9d4f2115d P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 4.
I20240502 14:01:16.298743 10374 raft_consensus.cc:2396] T a804b61491304055865347e70e8a9778 P 243b53aabd2d4443a8971e707bcc4a51 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 4 because replica has last-logged OpId of term: 3 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 10.
I20240502 14:01:16.299046 10180 leader_election.cc:302] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:16.299216 12499 raft_consensus.cc:2781] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Leader election won for term 4
I20240502 14:01:16.299305 12499 raft_consensus.cc:686] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [term 4 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.299252 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657170da057f4318a6165d2ed9097e58" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.299475 10374 raft_consensus.cc:3032] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.299437 12499 consensus_queue.cc:237] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.299568 10374 raft_consensus.cc:729] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.299793 10374 consensus_queue.cc:260] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.300022 10374 raft_consensus.cc:3037] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.300557 10245 leader_election.cc:288] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.301189 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.301311 10245 raft_consensus.cc:482] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.301379 10245 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.301551 10374 raft_consensus.cc:2396] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 14:01:16.302246 10245 raft_consensus.cc:504] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.302309 9965 catalog_manager.cc:5484] T 10bcbc135aa64d74bd5aaef9d4f2115d P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 3 to 4, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 4 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.302906 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.303026 10508 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.303155 10372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10bcbc135aa64d74bd5aaef9d4f2115d" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.303237 10372 raft_consensus.cc:3032] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 14:01:16.303270 10372 raft_consensus.cc:729] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.303377 10372 consensus_queue.cc:260] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.8, Last appended by leader: 8, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.303489 10372 raft_consensus.cc:3037] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.303998 10508 raft_consensus.cc:2396] T ef18a1e73625480a94c556f7960255a7 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 4 because replica has last-logged OpId of term: 2 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 14:01:16.304317 10245 leader_election.cc:288] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.304528 10372 raft_consensus.cc:2396] T 10bcbc135aa64d74bd5aaef9d4f2115d P 243b53aabd2d4443a8971e707bcc4a51 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 4 because replica has last-logged OpId of term: 3 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 7.
I20240502 14:01:16.305037 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aeceb15b637841609765e29e9b4d7ef1"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.305150 10245 raft_consensus.cc:482] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.305208 10245 raft_consensus.cc:3037] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.305729 11821 consensus_queue.cc:1035] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.305826 11821 consensus_queue.cc:1225] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Peer ee5d4bb5f8fa40d5813a5ee8bbfce3c0 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 14:01:16.306048 10245 raft_consensus.cc:504] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.306643 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeceb15b637841609765e29e9b4d7ef1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.306759 10114 raft_consensus.cc:3037] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.307796 10114 raft_consensus.cc:2454] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.308229 10180 leader_election.cc:302] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.308404 12117 raft_consensus.cc:2781] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.308387 11821 consensus_queue.cc:1035] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:16.308473 12117 raft_consensus.cc:686] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.308573 12117 consensus_queue.cc:237] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.309654 10178 leader_election.cc:302] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc; no voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0
I20240502 14:01:16.310249 12499 raft_consensus.cc:2726] T 0b80a9c354354433a48a0a5f8938265d P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 14:01:16.311426 9965 catalog_manager.cc:5484] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.313046 10508 pending_rounds.cc:76] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Aborting all ops after (but not including) 3
I20240502 14:01:16.313112 10508 pending_rounds.cc:97] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 14:01:16.313164 10508 raft_consensus.cc:2866] T 67cf72901dd0434d9b88902343d45977 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 14:01:16.313344 11821 consensus_queue.cc:1035] T 67cf72901dd0434d9b88902343d45977 P 243b53aabd2d4443a8971e707bcc4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.318259 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeceb15b637841609765e29e9b4d7ef1" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.318399 10508 raft_consensus.cc:3032] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.318447 10508 raft_consensus.cc:729] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.318620 10508 consensus_queue.cc:260] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.318719 10372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef18a1e73625480a94c556f7960255a7" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.318806 10508 raft_consensus.cc:3037] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.318821 10372 raft_consensus.cc:3032] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 14:01:16.318861 10372 raft_consensus.cc:729] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.318974 10372 consensus_queue.cc:260] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 3.15, Last appended by leader: 15, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.319092 10372 raft_consensus.cc:3037] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.319897 10508 raft_consensus.cc:2454] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.320506 10372 raft_consensus.cc:2396] T ef18a1e73625480a94c556f7960255a7 P 243b53aabd2d4443a8971e707bcc4a51 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 4 because replica has last-logged OpId of term: 3 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 14:01:16.320577 10245 leader_election.cc:288] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.321336 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "965596e659e340c7b231e4eef1868e92"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.321465 10245 raft_consensus.cc:482] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.321530 10245 raft_consensus.cc:3037] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.322507 10245 raft_consensus.cc:504] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.322860 10245 leader_election.cc:288] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.324033 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965596e659e340c7b231e4eef1868e92" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.324149 10114 raft_consensus.cc:3037] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.324427 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "965596e659e340c7b231e4eef1868e92" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.324522 10508 raft_consensus.cc:3032] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.324556 10508 raft_consensus.cc:729] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.324679 10508 consensus_queue.cc:260] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.324828 10508 raft_consensus.cc:3037] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.325153 10114 raft_consensus.cc:2454] T 965596e659e340c7b231e4eef1868e92 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.325817 10508 raft_consensus.cc:2454] T 965596e659e340c7b231e4eef1868e92 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.326296 10180 leader_election.cc:302] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:16.326562 12499 raft_consensus.cc:2781] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.326638 12499 raft_consensus.cc:686] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.326750 12499 consensus_queue.cc:237] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.328060 10178 leader_election.cc:302] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc; no voters: 243b53aabd2d4443a8971e707bcc4a51, ee5d4bb5f8fa40d5813a5ee8bbfce3c0
I20240502 14:01:16.328258 12117 raft_consensus.cc:2726] T ef18a1e73625480a94c556f7960255a7 P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 14:01:16.330188 9963 catalog_manager.cc:5484] T 965596e659e340c7b231e4eef1868e92 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.332208 10245 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3474ecec1b6f4fc29d6db00b8fa56b06"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.332350 10245 raft_consensus.cc:482] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20240502 14:01:16.332345 10465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51924: Illegal state: replica ee5d4bb5f8fa40d5813a5ee8bbfce3c0 is not leader of this config: current role FOLLOWER
I20240502 14:01:16.332407 10245 raft_consensus.cc:3037] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.333484 10245 raft_consensus.cc:504] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.338781 10508 raft_consensus.cc:1261] T aeceb15b637841609765e29e9b4d7ef1 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20240502 14:01:16.339401 10114 raft_consensus.cc:1261] T aeceb15b637841609765e29e9b4d7ef1 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20240502 14:01:16.339771 12117 consensus_queue.cc:1035] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:16.340315 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3474ecec1b6f4fc29d6db00b8fa56b06" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.340438 10112 raft_consensus.cc:3037] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.340720 12483 consensus_queue.cc:1035] T aeceb15b637841609765e29e9b4d7ef1 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:16.341581 10112 raft_consensus.cc:2454] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 2.
I20240502 14:01:16.341850 10180 leader_election.cc:302] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.341984 10245 leader_election.cc:288] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 2 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.342734 12483 raft_consensus.cc:2781] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:01:16.342839 12483 raft_consensus.cc:686] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [term 2 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.342885 10240 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.342979 12483 consensus_queue.cc:237] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.342979 10240 raft_consensus.cc:482] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.343173 10240 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.344070 10240 raft_consensus.cc:504] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.344413 10240 leader_election.cc:288] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.344795 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3474ecec1b6f4fc29d6db00b8fa56b06" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.344919 10508 raft_consensus.cc:3032] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:01:16.344964 10508 raft_consensus.cc:729] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.345002 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.345088 10508 consensus_queue.cc:260] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 10, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.345276 10508 raft_consensus.cc:3037] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:01:16.346217 10508 raft_consensus.cc:2454] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 2.
I20240502 14:01:16.345103 10507 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.346371 10240 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b0f4a67e31b04fefac38cf7d7cb983e7"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.346474 10240 raft_consensus.cc:482] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.346540 10240 raft_consensus.cc:3037] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.347259 10507 raft_consensus.cc:2454] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 4.
I20240502 14:01:16.347342 10240 raft_consensus.cc:504] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.347667 10240 leader_election.cc:288] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:16.348824 10180 leader_election.cc:302] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0; no voters:
I20240502 14:01:16.349006 12117 raft_consensus.cc:2781] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Leader election won for term 4
I20240502 14:01:16.349107 12117 raft_consensus.cc:686] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 4 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.349227 12117 consensus_queue.cc:237] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 3.10, Last appended by leader: 10, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.350544 10114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f4a67e31b04fefac38cf7d7cb983e7" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.350644 10114 raft_consensus.cc:3037] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.351681 10114 raft_consensus.cc:2454] T b0f4a67e31b04fefac38cf7d7cb983e7 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.351898 10240 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9c1b9c544e23474791c93f4094b4c672"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.352018 10240 raft_consensus.cc:482] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.352080 10240 raft_consensus.cc:3037] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.352183 10180 leader_election.cc:302] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.352376 12117 raft_consensus.cc:2781] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.352478 12117 raft_consensus.cc:686] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.352488 10114 raft_consensus.cc:1261] T 657170da057f4318a6165d2ed9097e58 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 14:01:16.352409 9963 catalog_manager.cc:5484] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 1 to 2, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 2 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.352583 12117 consensus_queue.cc:237] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.352669 9963 catalog_manager.cc:5484] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 3 to 4, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 4 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.352891 12483 consensus_queue.cc:1035] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.353073 10240 raft_consensus.cc:504] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.353384 10240 leader_election.cc:288] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293), 068f58913ea3478599df829d5118f7be (127.26.25.129:38571)
I20240502 14:01:16.353807 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1b9c544e23474791c93f4094b4c672" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.353925 10112 raft_consensus.cc:3037] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.354876 10112 raft_consensus.cc:2454] T 9c1b9c544e23474791c93f4094b4c672 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.355201 10375 raft_consensus.cc:1261] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 14:01:16.355783 10180 leader_election.cc:302] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.356040 12483 raft_consensus.cc:2781] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.356104 12483 raft_consensus.cc:686] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.356205 12483 consensus_queue.cc:237] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.357270 10372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1aa508f78bc40cdb8616017d9e0c5d8" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 4 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.357373 10372 raft_consensus.cc:3032] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 14:01:16.357412 10372 raft_consensus.cc:729] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.357558 10372 consensus_queue.cc:260] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 3.11, Last appended by leader: 11, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.357671 10372 raft_consensus.cc:3037] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.358726 10372 raft_consensus.cc:2396] T f1aa508f78bc40cdb8616017d9e0c5d8 P 243b53aabd2d4443a8971e707bcc4a51 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 4 because replica has last-logged OpId of term: 3 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 10.
I20240502 14:01:16.358891 10240 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f0b83b1163014375a53d97d95f4fcb13"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.358991 10240 raft_consensus.cc:482] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.359045 10240 raft_consensus.cc:3037] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.359912 10240 raft_consensus.cc:504] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.360044 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1b9c544e23474791c93f4094b4c672" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.360173 10507 raft_consensus.cc:3032] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.360213 10507 raft_consensus.cc:729] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.360246 10240 leader_election.cc:288] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:16.360347 10507 consensus_queue.cc:260] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.360468 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b83b1163014375a53d97d95f4fcb13" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.360520 10507 raft_consensus.cc:3037] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.360560 10113 raft_consensus.cc:3037] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.361429 10507 raft_consensus.cc:2454] T 9c1b9c544e23474791c93f4094b4c672 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.361429 10113 raft_consensus.cc:2454] T f0b83b1163014375a53d97d95f4fcb13 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.361742 10180 leader_election.cc:302] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.361943 12499 raft_consensus.cc:2781] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.362035 12499 raft_consensus.cc:686] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.362092 10240 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a54336e39ef249fdb7eb6b50cb5b1cda"
dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
from {username='slave'} at 127.0.0.1:52000
I20240502 14:01:16.362155 12499 consensus_queue.cc:237] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.362196 10240 raft_consensus.cc:482] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.362258 10240 raft_consensus.cc:3037] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.363065 10240 raft_consensus.cc:504] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.363396 10240 leader_election.cc:288] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Requested vote from peers 068f58913ea3478599df829d5118f7be (127.26.25.129:38571), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.364075 10113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54336e39ef249fdb7eb6b50cb5b1cda" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "068f58913ea3478599df829d5118f7be"
I20240502 14:01:16.364177 10113 raft_consensus.cc:3037] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.365015 10113 raft_consensus.cc:2454] T a54336e39ef249fdb7eb6b50cb5b1cda P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.365105 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54336e39ef249fdb7eb6b50cb5b1cda" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.365203 10507 raft_consensus.cc:3032] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.365245 10507 raft_consensus.cc:729] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ee5d4bb5f8fa40d5813a5ee8bbfce3c0, State: Running, Role: LEADER
I20240502 14:01:16.365487 10507 consensus_queue.cc:260] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.365662 10507 raft_consensus.cc:3037] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.365931 10180 leader_election.cc:302] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 1ca5103abf454aaca375e9e07eb139cc; no voters:
I20240502 14:01:16.366097 12483 raft_consensus.cc:2781] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.366194 12483 raft_consensus.cc:686] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Leader. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.366324 12483 consensus_queue.cc:237] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 10, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.368275 10507 raft_consensus.cc:2454] T a54336e39ef249fdb7eb6b50cb5b1cda P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.369042 9963 catalog_manager.cc:5484] T b0f4a67e31b04fefac38cf7d7cb983e7 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.369342 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f4a67e31b04fefac38cf7d7cb983e7" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.369452 10374 raft_consensus.cc:3032] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.369499 10374 raft_consensus.cc:729] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.369634 10374 consensus_queue.cc:260] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.369805 10374 raft_consensus.cc:3037] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.370342 9963 catalog_manager.cc:5484] T 9c1b9c544e23474791c93f4094b4c672 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.371742 10374 raft_consensus.cc:2454] T b0f4a67e31b04fefac38cf7d7cb983e7 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca5103abf454aaca375e9e07eb139cc in term 3.
I20240502 14:01:16.373214 12117 consensus_queue.cc:1035] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:16.373337 12117 consensus_queue.cc:1225] T 657170da057f4318a6165d2ed9097e58 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Peer 243b53aabd2d4443a8971e707bcc4a51 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 14:01:16.373579 10240 raft_consensus.cc:1226] T f1aa508f78bc40cdb8616017d9e0c5d8 P 1ca5103abf454aaca375e9e07eb139cc [term 4 LEADER]: Rejecting Update request from peer 243b53aabd2d4443a8971e707bcc4a51 for earlier term 3. Current term is 4. Ops: [3.11-3.11]
I20240502 14:01:16.374267 10507 raft_consensus.cc:1226] T f1aa508f78bc40cdb8616017d9e0c5d8 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 4 FOLLOWER]: Rejecting Update request from peer 243b53aabd2d4443a8971e707bcc4a51 for earlier term 3. Current term is 4. Ops: [3.11-3.11]
W20240502 14:01:16.375094 10465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51880: Illegal state: replica ee5d4bb5f8fa40d5813a5ee8bbfce3c0 is not leader of this config: current role FOLLOWER
I20240502 14:01:16.375059 9963 catalog_manager.cc:5484] T f0b83b1163014375a53d97d95f4fcb13 P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.375295 9963 catalog_manager.cc:5484] T a54336e39ef249fdb7eb6b50cb5b1cda P 1ca5103abf454aaca375e9e07eb139cc reported cstate change: term changed from 2 to 3, leader changed from ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132) to 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130). New cstate: current_term: 3 leader_uuid: "1ca5103abf454aaca375e9e07eb139cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:16.380398 10372 pending_rounds.cc:76] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51: Aborting all ops after (but not including) 3
I20240502 14:01:16.380476 10372 pending_rounds.cc:97] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 14:01:16.380539 10372 raft_consensus.cc:2866] T 657170da057f4318a6165d2ed9097e58 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20240502 14:01:16.382525 10073 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39034: Illegal state: replica 068f58913ea3478599df829d5118f7be is not leader of this config: current role FOLLOWER
I20240502 14:01:16.382979 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0b83b1163014375a53d97d95f4fcb13" candidate_uuid: "1ca5103abf454aaca375e9e07eb139cc" candidate_term: 3 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.383082 10374 raft_consensus.cc:3032] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.383116 10374 raft_consensus.cc:729] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.383246 10374 consensus_queue.cc:260] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.9, Last appended by leader: 9, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.383392 10374 raft_consensus.cc:3037] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.384941 10114 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8"
dest_uuid: "068f58913ea3478599df829d5118f7be"
from {username='slave'} at 127.0.0.1:39094
I20240502 14:01:16.385073 10114 raft_consensus.cc:482] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.385115 10114 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.386030 10114 raft_consensus.cc:504] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:16.385663 10374 raft_consensus.cc:2396] T f0b83b1163014375a53d97d95f4fcb13 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca5103abf454aaca375e9e07eb139cc for term 3 because replica has last-logged OpId of term: 2 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 8.
I20240502 14:01:16.386394 10114 leader_election.cc:288] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 4 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:16.386991 10374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.387097 10374 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.387213 10114 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c958a9027a064dcb9026608541a22883"
dest_uuid: "068f58913ea3478599df829d5118f7be"
from {username='slave'} at 127.0.0.1:39094
I20240502 14:01:16.387198 10240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe234bfab79843d79b0ce5228e678ab8" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.387302 10114 raft_consensus.cc:482] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.387305 10240 raft_consensus.cc:3032] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 14:01:16.387349 10240 raft_consensus.cc:729] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.387377 10114 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.387492 10240 consensus_queue.cc:260] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } }
I20240502 14:01:16.387642 10240 raft_consensus.cc:3037] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.388108 10374 raft_consensus.cc:2396] T fe234bfab79843d79b0ce5228e678ab8 P 243b53aabd2d4443a8971e707bcc4a51 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 068f58913ea3478599df829d5118f7be for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 14:01:16.388289 10114 raft_consensus.cc:504] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.388689 10114 leader_election.cc:288] T c958a9027a064dcb9026608541a22883 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 4 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.388720 10240 raft_consensus.cc:2396] T fe234bfab79843d79b0ce5228e678ab8 P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 068f58913ea3478599df829d5118f7be for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 14:01:16.388906 10372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c958a9027a064dcb9026608541a22883" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.388995 10372 raft_consensus.cc:3032] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 14:01:16.389310 10114 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0577d6cbf5a04722843d7ecf9cf28be9"
dest_uuid: "068f58913ea3478599df829d5118f7be"
from {username='slave'} at 127.0.0.1:39094
I20240502 14:01:16.389420 10114 raft_consensus.cc:482] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.389482 10114 raft_consensus.cc:3037] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.389031 10372 raft_consensus.cc:729] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.389779 10372 consensus_queue.cc:260] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.389866 10508 raft_consensus.cc:1261] T 3474ecec1b6f4fc29d6db00b8fa56b06 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20240502 14:01:16.389925 10372 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.390282 10114 raft_consensus.cc:504] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.390820 10370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0577d6cbf5a04722843d7ecf9cf28be9" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 3 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.390928 10370 raft_consensus.cc:3037] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.390908 12117 consensus_queue.cc:1035] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:16.391780 10370 raft_consensus.cc:2396] T 0577d6cbf5a04722843d7ecf9cf28be9 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 068f58913ea3478599df829d5118f7be for term 3 because replica has last-logged OpId of term: 1 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 14:01:16.392928 10049 leader_election.cc:302] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 068f58913ea3478599df829d5118f7be; no voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51
I20240502 14:01:16.393160 10112 raft_consensus.cc:1261] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20240502 14:01:16.393342 12127 raft_consensus.cc:2726] T fe234bfab79843d79b0ce5228e678ab8 P 068f58913ea3478599df829d5118f7be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 14:01:16.393688 10114 leader_election.cc:288] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 3 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739)
I20240502 14:01:16.393736 12117 consensus_queue.cc:1035] T 3474ecec1b6f4fc29d6db00b8fa56b06 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:16.394054 10240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0577d6cbf5a04722843d7ecf9cf28be9" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 3 candidate_status { last_received { term: 1 index: 7 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.394153 10240 raft_consensus.cc:3032] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.394194 10240 raft_consensus.cc:729] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.394335 10240 consensus_queue.cc:260] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } }
I20240502 14:01:16.394475 10240 raft_consensus.cc:3037] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.395502 10240 raft_consensus.cc:2396] T 0577d6cbf5a04722843d7ecf9cf28be9 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 068f58913ea3478599df829d5118f7be for term 3 because replica has last-logged OpId of term: 2 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 7.
I20240502 14:01:16.396502 10049 leader_election.cc:302] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 068f58913ea3478599df829d5118f7be; no voters: 1ca5103abf454aaca375e9e07eb139cc, 243b53aabd2d4443a8971e707bcc4a51
I20240502 14:01:16.396642 12127 raft_consensus.cc:2726] T 0577d6cbf5a04722843d7ecf9cf28be9 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20240502 14:01:16.398779 10465 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51880: Illegal state: replica ee5d4bb5f8fa40d5813a5ee8bbfce3c0 is not leader of this config: current role FOLLOWER
I20240502 14:01:16.401809 10372 raft_consensus.cc:2396] T c958a9027a064dcb9026608541a22883 P 243b53aabd2d4443a8971e707bcc4a51 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 068f58913ea3478599df829d5118f7be for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 14:01:16.402004 10114 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2379dbd40b744991be25c2914f878913"
dest_uuid: "068f58913ea3478599df829d5118f7be"
from {username='slave'} at 127.0.0.1:39094
I20240502 14:01:16.402118 10114 raft_consensus.cc:482] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.402176 10114 raft_consensus.cc:3037] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.403203 10114 raft_consensus.cc:504] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.403728 10507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c958a9027a064dcb9026608541a22883" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.403879 10507 raft_consensus.cc:3037] T c958a9027a064dcb9026608541a22883 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Advancing to term 4
W20240502 14:01:16.403955 10073 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39034: Illegal state: replica 068f58913ea3478599df829d5118f7be is not leader of this config: current role FOLLOWER
I20240502 14:01:16.404196 10372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2379dbd40b744991be25c2914f878913" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.404311 10372 raft_consensus.cc:3032] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 14:01:16.404362 10372 raft_consensus.cc:729] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 243b53aabd2d4443a8971e707bcc4a51, State: Running, Role: LEADER
I20240502 14:01:16.404500 10372 consensus_queue.cc:260] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.405858 10372 raft_consensus.cc:3037] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.406863 10372 raft_consensus.cc:2454] T 2379dbd40b744991be25c2914f878913 P 243b53aabd2d4443a8971e707bcc4a51 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 3.
I20240502 14:01:16.407228 10114 leader_election.cc:288] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 3 election: Requested vote from peers 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.407526 10047 leader_election.cc:302] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f58913ea3478599df829d5118f7be, 243b53aabd2d4443a8971e707bcc4a51; no voters:
I20240502 14:01:16.407682 12127 raft_consensus.cc:2781] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Leader election won for term 3
I20240502 14:01:16.407773 12127 raft_consensus.cc:686] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [term 3 LEADER]: Becoming Leader. State: Replica: 068f58913ea3478599df829d5118f7be, State: Running, Role: LEADER
I20240502 14:01:16.407925 12127 consensus_queue.cc:237] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.408886 10112 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49"
dest_uuid: "068f58913ea3478599df829d5118f7be"
from {username='slave'} at 127.0.0.1:39094
I20240502 14:01:16.408989 10112 raft_consensus.cc:482] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.409049 10112 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.409993 10112 raft_consensus.cc:504] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.410298 10112 leader_election.cc:288] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 4 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), ee5d4bb5f8fa40d5813a5ee8bbfce3c0 (127.26.25.132:38293)
I20240502 14:01:16.411571 9965 catalog_manager.cc:5484] T 2379dbd40b744991be25c2914f878913 P 068f58913ea3478599df829d5118f7be reported cstate change: term changed from 2 to 3, leader changed from 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131) to 068f58913ea3478599df829d5118f7be (127.26.25.129). New cstate: current_term: 3 leader_uuid: "068f58913ea3478599df829d5118f7be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.412586 10508 raft_consensus.cc:1261] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Refusing update from remote peer 1ca5103abf454aaca375e9e07eb139cc: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240502 14:01:16.412911 12483 consensus_queue.cc:1035] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:16.412974 10112 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1"
dest_uuid: "068f58913ea3478599df829d5118f7be"
from {username='slave'} at 127.0.0.1:39094
I20240502 14:01:16.413070 10112 raft_consensus.cc:482] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.413125 10112 raft_consensus.cc:3037] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.414067 10112 raft_consensus.cc:504] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.414809 10112 leader_election.cc:288] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 4 election: Requested vote from peers 1ca5103abf454aaca375e9e07eb139cc (127.26.25.130:39739), 243b53aabd2d4443a8971e707bcc4a51 (127.26.25.131:40365)
I20240502 14:01:16.415635 10508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2379dbd40b744991be25c2914f878913" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.415731 10508 raft_consensus.cc:3037] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 2 FOLLOWER]: Advancing to term 3
I20240502 14:01:16.415915 10240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 4 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.416023 10240 raft_consensus.cc:3032] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 14:01:16.416069 10240 raft_consensus.cc:729] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.416110 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec3efe6d5bc45a79458230c43f26b49" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 4 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0"
I20240502 14:01:16.416194 10506 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.416196 10240 consensus_queue.cc:260] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.10, Last appended by leader: 10, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "ee5d4bb5f8fa40d5813a5ee8bbfce3c0" member_type: VOTER last_known_addr { host: "127.26.25.132" port: 38293 } }
I20240502 14:01:16.416308 10240 raft_consensus.cc:3037] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.416637 10508 raft_consensus.cc:2454] T 2379dbd40b744991be25c2914f878913 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f58913ea3478599df829d5118f7be in term 3.
I20240502 14:01:16.417104 10506 raft_consensus.cc:2396] T 3ec3efe6d5bc45a79458230c43f26b49 P ee5d4bb5f8fa40d5813a5ee8bbfce3c0 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 068f58913ea3478599df829d5118f7be for term 4 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
I20240502 14:01:16.417205 10240 raft_consensus.cc:2396] T 3ec3efe6d5bc45a79458230c43f26b49 P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 068f58913ea3478599df829d5118f7be for term 4 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 8.
I20240502 14:01:16.417704 10240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 4 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "1ca5103abf454aaca375e9e07eb139cc"
I20240502 14:01:16.417755 10049 leader_election.cc:302] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 068f58913ea3478599df829d5118f7be; no voters: 1ca5103abf454aaca375e9e07eb139cc, ee5d4bb5f8fa40d5813a5ee8bbfce3c0
I20240502 14:01:16.417789 10240 raft_consensus.cc:3032] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Stepping down as leader of term 3
I20240502 14:01:16.417822 10240 raft_consensus.cc:729] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca5103abf454aaca375e9e07eb139cc, State: Running, Role: LEADER
I20240502 14:01:16.417925 11849 raft_consensus.cc:2726] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 14:01:16.417934 10240 consensus_queue.cc:260] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.9, Last appended by leader: 9, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131" port: 40365 } }
I20240502 14:01:16.418040 10240 raft_consensus.cc:3037] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.418133 10111 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bb0d4740d5ca4d80a80751151b3f57a7"
dest_uuid: "068f58913ea3478599df829d5118f7be"
from {username='slave'} at 127.0.0.1:39094
I20240502 14:01:16.418195 10112 raft_consensus.cc:1226] T 3ec3efe6d5bc45a79458230c43f26b49 P 068f58913ea3478599df829d5118f7be [term 4 FOLLOWER]: Rejecting Update request from peer 1ca5103abf454aaca375e9e07eb139cc for earlier term 3. Current term is 4. Ops: []
I20240502 14:01:16.418242 10111 raft_consensus.cc:482] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:01:16.418294 10111 raft_consensus.cc:3037] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 3 FOLLOWER]: Advancing to term 4
I20240502 14:01:16.418519 10372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9325aa99d6d48d2bcbc39ffdce77bf1" candidate_uuid: "068f58913ea3478599df829d5118f7be" candidate_term: 4 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "243b53aabd2d4443a8971e707bcc4a51"
I20240502 14:01:16.418882 10240 raft_consensus.cc:2396] T d9325aa99d6d48d2bcbc39ffdce77bf1 P 1ca5103abf454aaca375e9e07eb139cc [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 068f58913ea3478599df829d5118f7be for term 4 because replica has last-logged OpId of term: 3 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 8.
I20240502 14:01:16.419201 10111 raft_consensus.cc:504] T bb0d4740d5ca4d80a80751151b3f57a7 P 068f58913ea3478599df829d5118f7be [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca5103abf454aaca375e9e07eb139cc" member_type: VOTER last_known_addr { host: "127.26.25.130" port: 39739 } } peers { permanent_uuid: "068f58913ea3478599df829d5118f7be" member_type: VOTER last_known_addr { host: "127.26.25.129" port: 38571 } } peers { permanent_uuid: "243b53aabd2d4443a8971e707bcc4a51" member_type: VOTER last_known_addr { host: "127.26.25.131"